Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th January 2014, 09:08   #521  |  Link
dvdan23
Registered User
 
dvdan23's Avatar
 
Join Date: Feb 2007
Posts: 4
Turbopascal7, I certainly wouldn't want Myrsloik to kill you. I also don't want to see Linux compatibility killed though.

As I missed this discussion, I take it "requiring a huge dependency" was deemed the greater evil vs. breaking under Linux? FWIW, that wouldn't have been my vote.

If boost libraries are indeed of comparable quality, it seems to me it's a dependency either way. In my case, the dependency of windows is a far greater evil than the dependency on a free, portable library like boost. the boost licensing is pretty unrestrictive. How's the performance?

If the death of Avisynth+ on Linux is truly unavoidable, and last minute calls to the governor can't do anything to save it, might it be feasible to keep a single-threaded branch alive on the git hub and/or make a periodic build for folks who can't run VC2012++? if other solutions pan out that'll be great but I suspect that might be a few years.
dvdan23 is offline   Reply With Quote
Old 24th January 2014, 09:40   #522  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by dvdan23 View Post
If boost libraries are indeed of comparable quality, it seems to me it's a dependency either way. In my case, the dependency of windows is a far greater evil than the dependency on a free, portable library like boost. the boost licensing is pretty unrestrictive. How's the performance?
Dependency on windows has always been there, nothing really changed on that front. The fact that wine has bugs is, well, a wine's problem. I don't think it's appropriate to make life of 99% of avisynth users and all contributors harder to compensate for it. Dependencies on external libraries over standard are pretty much always worse - it's harder to setup build enviroment, it takes longer to compile and usually produces large binaries.

I understand that it's quite unfortunate that avs+ won't work on linux, but it can't be helped. You could always setup a VM too. And if the demand for linux support is high, of course there will be some developers willing to port plugins to this platform.

Quote:
Originally Posted by dvdan23 View Post
If the death of Avisynth+ on Linux is truly unavoidable, and last minute calls to the governor can't do anything to save it, might it be feasible to keep a single-threaded branch alive on the git hub and/or make a periodic build for folks who can't run VC2012++? if other solutions pan out that'll be great but I suspect that might be a few years.
Not to mention that it might be quite hard to merge the changes, there's one more problem with this solution - the plugins. With MT implemented in avs+ core, internal threading in plugins is strongly discouraged and you can expect it to get removed from most "updated" plugins, leading a lot worse performance with singlethreaded versions of avisynth. Asking developers to maintain branches with internally threaded code in all of these plugins is way too much. So you'll be either enjoying crippled performance or will have to avoid newer versions of such plugins.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 24th January 2014, 09:57   #523  |  Link
dvdan23
Registered User
 
dvdan23's Avatar
 
Join Date: Feb 2007
Posts: 4
Learned something new from the Wine folks. Apparently the approach they are taking is to build a complete VC++ library replacement so you don't need to install VC++ runtimes. Apparently if the "Microsoft Foundation Classes" can be avoided, there's a better shot at Avisynth++ working under WINE.

From the bug $17273 "Applications affected" page;

NOTE: There is an ongoing effort in Wine to provide a complete Microsoft Visual C++runtime library replacement - removing the need of installing the Microsoft VC++ redistributable packages. This is a work in progress hence you might still need to install Microsoft VC++ runtime.

If the application uses MFC library (Microsoft Foundation Classes), there will be no replacement hence installation of Microsoft Redistributable is a requirement for applications and games to work!

TurboPascal7, I'm not going to argue with you because I'm thrilled with the work you are doing making more plugins 64-bit. I'll tell you though, I'm much more interested in stable, working, 64-bit non-threaded avisynth plugins than threaded ones. Keep up the good work.
dvdan23 is offline   Reply With Quote
Old 24th January 2014, 10:02   #524  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
The only thing I wanted to say is that making the wine guys fix their stuff is a lot better than limiting program functionality in any way. I'm sure if users complain enough, they'll get it working sooner than later.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 25th January 2014, 01:22   #525  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,057
It would appear that a deeper fix would be needed anyway, since while it is possible to yank the .dlls themselves out of the vcredist installer*, and thereby avoid the Wine errors concerning Apartment Threading when trying to install normally, the .dlls either need extra registering during the install process, or Wine has a problem with them regardless of the installer and needs fixing (which is also likely). Trying to use them with AviSynth+ and ffmpeg resulted in ffmpeg crashing with a C++ exception of some sort.

Granted, I was testing with Visual Studio 2013 builds and the corresponding vcredist, but I'm sure the same things apply for 2012 (or maybe they don't and for 2012 this would actually work; I don't know).


*Methodology ('dark' is dark.exe from the WiX Toolset):
Code:
dark vcredist2013.exe -x outputpath
go into vcRuntimeMinimum_x86 and use cabextract or 7-zip on the cab files; the MFC stuff is in vcRuntimeAdditional_x86
rename the F_CENTRAL files to .dll with the proper names (trim off the F_CENTRAL prefix and arch suffix, add .dll extension)
copy dlls to Wine's windows/system32 directory.
qyot27 is offline   Reply With Quote
Old 30th January 2014, 14:21   #526  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Ahh, d9 is back again. So in case if you want to know what is happening with AviSynth+: A lot!

First of all, we had a very long meeting last weekend where we focused on the most pressing matters about our GSoC application. Some still need finalization and most importantly, the ideas still need to be published in a well-formulated form. Also, please welcome our GSoC team. SEt, TurboPascal7, and myself are official mentors, while Innocenat and Firesledge have offered to help us from behind the scenes during GSoC. We'll let you know more about GSoC projects for Avs+ soon.

The MT capability of AviSynth+ is coming along nicely too, and after some more pushes from me the others have starting looking at making all the internal filters MT-safe. There is also a new API that is noteworthy, because it allows lower memory usage in general (even in single-threaded scripts!) for all filters that reserve memory on construction. Filters are being modified to make use of that too. After some bug fixes the MT build seems pretty stable, but the performance is not at its top yet. Nevertheless, this weekend or shortly after that there will probably be a public test, for the interested amongst you.

Also, I've heard today that Innocenat is experimenting with a Linux port of AviSynth+. Don't expect it to come in a couple of days since he only started recently, but work has been started nonetheless.

One kind of help we could use is if somebody stepped up to maintain the website. There will be more pages to post there real soon, so the current layout won't cut it for long. Not to mention the code duplication between pages currently isn't exactly maintainer-friendly. So it'd be nice if one of you could alter it or come forward with a new design, and maybe make changes in the future when we have new content to post. So let us know if you can help here.

Cheer for us to be successfull, we'll be back with more updates later. And don't forget to star us on GitHub
__________________
AviSynth+

Last edited by ultim; 30th January 2014 at 20:47.
ultim is offline   Reply With Quote
Old 30th January 2014, 23:31   #527  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
Does this mean Innocenat is done working on tivtc?

If so, was there no gains or something else?
__________________
PC: FX-8320 GTS250 HTPC: G1610 GTX650
PotPlayer/MPC-BE LAVFilters MadVR-Bicubic75AR/Lanczos4AR/Lanczos4AR LumaSharpen -Strength0.9-Pattern3-Clamp0.1-OffsetBias2.0
turbojet is offline   Reply With Quote
Old 30th January 2014, 23:33   #528  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by turbojet View Post
Does this mean Innocenat is done working on tivtc?
No, it doesn't.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 31st January 2014, 00:15   #529  |  Link
innocenat
Registered User
 
innocenat's Avatar
 
Join Date: Dec 2011
Posts: 77
Quote:
Originally Posted by turbojet View Post
Does this mean Innocenat is done working on tivtc?

If so, was there no gains or something else?
No, I am not.

There are times when I have free time with only Linux available, so I use that time to port.
__________________
AviSynth+
innocenat is offline   Reply With Quote
Old 31st January 2014, 05:41   #530  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
I see, thanks for the update, eagerly anticipating a faster tivtc or a wip ivtc filter that's just as capable. Maybe you've seen it but tritical's post may help.
__________________
PC: FX-8320 GTS250 HTPC: G1610 GTX650
PotPlayer/MPC-BE LAVFilters MadVR-Bicubic75AR/Lanczos4AR/Lanczos4AR LumaSharpen -Strength0.9-Pattern3-Clamp0.1-OffsetBias2.0
turbojet is offline   Reply With Quote
Old 2nd February 2014, 21:26   #531  |  Link
Paser
Registered User
 
Join Date: Jan 2014
Posts: 15
When I try to replace the AviSynth.dll in SysWOW64 with my newly compiled one, I get this error after trying to load an avs in VirtualDub:



Any way to fix this?
Paser is offline   Reply With Quote
Old 2nd February 2014, 21:29   #532  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,130
Maybe you should post your script...
Stereodude is offline   Reply With Quote
Old 2nd February 2014, 22:44   #533  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by Paser View Post
When I try to replace the AviSynth.dll in SysWOW64 with my newly compiled one, I get this error after trying to load an avs in VirtualDub:



Any way to fix this?
The current builds definetely work with VirtualDub. Are you using the correct architecture? E.g. if you are placing the plugin in SysWOW64, you must be running 32-bit VirtualDub, and you must use the 32-bit version of AviSynth.

If all else fails, try running AviSynth+'s setup, that does the upgrade for you correctly. But given that you take the correct DLL versions, just replacing the avisynth.dll should also be enough.
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 4th February 2014, 08:33   #534  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 3,192
Is it stable yet? I been waiting for MT build for months. I has the itch, go go go!
ryrynz is offline   Reply With Quote
Old 4th February 2014, 08:36   #535  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
You can always build it yourself or join our IRC channel (#avs-plus on Rizon) if you want to test things before they make it to this forum.
MT is not fully "stable" yet, but test builds will follow very soon.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 5th February 2014, 18:46   #536  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
OK, sorry that I missed the release on the weekend, but it is not easy to stay on time with lots of things going on, like overtime and deadlines at work, private server upgrade (which must also be done on deadline or else I'll loose my data), internet outage, lot's of GSoC preparations, and even (yes) some coding for AviSynth, to name a few from the top of my head.

But I bring you now a shiny new test build, r1689, which - in contrast to the previous one - is actually usable. Probably even much better than that. So I strongly suggest everyone to give it a shot, aside that I can probably improve on the thread scheduler for some more performance, you'll have fun with it (in a good way). Just make sure you set the correct MT mode for your filters. Here is a snippet that you can start with, but please add some new filters to that list on your own too. Don't bother with built-in filters though, they're already handled internally, so you only need to add filters from external plugins.

The MT branch is now also the main branch of AviSynth+, which means all (even non-MT) improvements end up here, and it will be merged into "master" as soon as it has received enough testing. But now you might wonder what are the "other" user-visible changes compared to the stable release. Mainly:
  • innocenat has worked more on the resizers, which are now even faster and require less memory, especially (but not only) when working on planar video and you have SSE3.
  • This issue is fixed, which sometimes caused that a filter function from the wrong DLL would get used. Thx for reporting the issue, real.finder.
  • If a plugin DLL cannot be loaded, a human-readable error from Windows is also displayed, giving the user a clue what is wrong.
  • Filters that reserve memory on construction can now share that same piece of memory between multiple instances, giving large memory savings in many cases. Most affected internal filters have been updated to make use of this capability, most by tp7. Some external filters will follow when the API is officially stable.
  • The new caching system is done, and it will result in noticably lower memory usage than any previous "classical" AviSynth version. Give it a try, you'll be surprised how much memory it brings in complex scripts.
  • SSE2 has been a requirement for Avs+ by mistake, this is fixed now, and you now only need an SSE-machine.
Of course all the above is paired with a lot of rewrites, refactorings, and cleanups. And then there's MT.

"Yeah-yeah-yeah, enough with the talk, just give me a build already." Right, http://goo.gl/e0VFYn. And don't forget to add some filters (after testing them) yourself to this list: https://pad.riseup.net/p/avs_plus_mt_modes

In my next post I'll briefly summarize how to use AviSynth+ in its multithreaded mode.
__________________
AviSynth+

Last edited by ultim; 6th February 2014 at 09:51.
ultim is offline   Reply With Quote
Old 5th February 2014, 19:27   #537  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
So, how to use MT in AviSynth+? Most of it has been posted earlier actually, but let me summarize it.

By default, your script will run in single-threaded mode, just like with SEt's build. Also, just like in SEt's build, you'll have to make sure that filters use the correct MT mode, or else they might wreak havoc. There are three MT modes (1,2,3), and they are the same modes as in (yeah you guessed correctly) SEt's build. Which means you can use the same modes that you have used with AviSynth-MT.

There are some things though that are different and/or new in AviSynth+. The first difference is *how* you set the MT mode. In AviSynth-MT, you had to use SetMTMode(X), which caused all filters following that line to use mode X (until the next call to SetMTMode()). This meant if you needed to use multiple MT modes, you had to insert all those calls in the middle of your script, littered over many places.

Setting MT modes
AviSynth+ does it differently. In AviSynth+, you specify the MT-mode for only specific filters, and those filters will then automatically use their own mode, even if there were other MT-modes inbetween. This means you can specify all the MT modes at the beginning without polluting your script. You can even make a SetMTMode.avsi if you wish and let it autoload for all of your scripts, or import() it from their top. This is much cleaner, and it allows you to maintain all your MT-modes centrally at a single place. To make this distinction clear from AviSynth+, SetMTMode() is called SetFilterMTMode() in AviSynth+.

Enabling MT
The other difference is how you actually enable multithreading. Calling SetFilterMTMode() is not enough, it sets the MT mode, but the MT mode only has an effect if MT is enabled at all. Note this means you can safely include/import/autoload your SetFilterMTMode() calls in even single-threaded scripts, and they will not be messed up. Uhm, onto the point: You enable MT by placing a single call to Prefetch(X) at the *end* of your script, where X is the number of threads to use.

Example
Code:
# This line causes all filters that don't have an MT mode explicitly use mode 2 by default.
# Mode 2 is a relatively safe choice until you don't know most of your calls to be either mode 1 or 3.
# Compared with mode 1, mode 2 trades memory for MT-safety, but only a select few filters will work with mode 1.
SetFilterMTMode("DEFAULT_MT_MODE", 2)

# FFVideoSource(), like most source filters, needs MT mode 3
SetFilterMTMode("FFVideoSource", 3)

# Now comes your script as usual
FFVideoSource(...)
trim(...)
MCTemporalDenoise(...)
...

# Enable MT!
Prefetch(4)
Closing notes (don't skip!)
  • Remember that MT is only stable as long as you have specified a correct MT mode for all filters.
  • Instead of the numbers 1-2-3, you can also use symbolic names for MT modes: MT_NICE_FILTER (1), MT_MULTI_INSTANCE (2), MT_SERIALIZED (3)
  • Mode 3 is evil. It is necessary for some filters, and it is usually no problem for source filters, but it can literally completely negate all advantages of MT, if such a filter is placed near the end of your script. Let us know if you meet a non-source mode 3 filter, we might be able to do something about it, but in general, avoid such calls if you want performance. (And of course, insert what you have found into here.)
  • The new caches will save you a lot of memory in single-threaded scripts, but due to the way they work, they will also use more memory than before with MT enabled. The memory usage will scale much closer with the number of threads you have. Just something to keep in mind.
  • MT-enabled AviSynth+ triggers a latent bug in AvsPmod. Until a new version of AvsPmod is officially released, use this build. A thousand thanks to vdcrim for the fix.
  • Using too many threads can easily hurt performance a lot, because there are other bottlenecks too in your PC than just the CPU. For example, if you have a quad-core machine with 8 logical cores, less than 8 threads will often work much better than 8 or more.
__________________
AviSynth+

Last edited by ultim; 16th March 2015 at 10:23. Reason: Updated instructions for newest build
ultim is offline   Reply With Quote
Old 5th February 2014, 20:10   #538  |  Link
easyfab
Registered User
 
Join Date: Jan 2002
Posts: 327
Thanks Ultim and avs+ team for this release, here my first little test on an intel I2600K.

http://pastebin.com/Di0w2Y0y

Not bad for a first version ,
for info avs+ single thread -> FPS (min | max | average): 28.47 | 32.69 | 30.88

The difference I see is in frame count 145 vs 103, could it explain the difference in speed ?
easyfab is offline   Reply With Quote
Old 5th February 2014, 20:11   #539  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
The old masktools2-a48 has a serious issue that will make things blow up/crash if MT is used. Newer rebuilds of that same plugin work correctly, such as the one from QTGMC's Modded Plugin Package, or TurboPascal7's rewrite.
__________________
AviSynth+

Last edited by ultim; 5th February 2014 at 20:25.
ultim is offline   Reply With Quote
Old 5th February 2014, 20:21   #540  |  Link
easyfab
Registered User
 
Join Date: Jan 2002
Posts: 327
I have mt_masktools-26.dll (from 06_taro ) and I try with TurboPascal7's rewrite It give me the ~ same result -> FPS (min | max | average): 33.20 | 144.55 | 74.46

Last edited by easyfab; 5th February 2014 at 20:23.
easyfab is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.