Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 13th October 2013, 18:10   #141  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth. So I have to the choice of 1) either keeping the original, or 2) merging features from ffms2 but breaking C-compatibility to upstream.

As a sad note, the C-interface has a bunch of baked code. What surprises me is that Avisynth now has three different APIs (2.5, 2.6, and C), and all of them suck. I honestly feel tempted to finally create an API for Avisynth that 1) doesn't rely on the representation of internal data structures, 2) doesn't rely on baked code, and 3) can be used universally from many different compilers and computer languages (right now whichever existing interface you choose, you can pick at most only one of these properties). I'm just afraid that XKCD #927 will set in.
ultim is offline  
Old 13th October 2013, 18:36   #142  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by ultim View Post
I'm just afraid that XKCD #927 will set in.
Or this.
Groucho2004 is offline  
Old 13th October 2013, 19:00   #143  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by ultim View Post
qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth. So I have to the choice of 1) either keeping the original, or 2) merging features from ffms2 but breaking C-compatibility to upstream.
As it was a partial merge of 2.6 features into it, I'm not surprised (compatibility with the upstream's 2.5-centric C interface was relegated to a secondary header that defined old versions of avs_get_row_size_p and avs_get_height_p). The intent was probably for the upstream to catch up to these features whenever 2.6 finalized, or something.

Having a new API is likely going to be needed regardless, since the introduction of really-new or novel features would probably break the existing one anyway. Might as well develop the new features against an API that's cleaner and easier to maintain, IMO. It'd also be an opportunity to document it better, if all the complaints I've heard over the years about how poorly documented the existing one is is any indication (noble efforts to rectify this notwithstanding).

Last edited by qyot27; 13th October 2013 at 22:49. Reason: fix typo
qyot27 is offline  
Old 13th October 2013, 20:56   #144  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
qyot27: I looked at the avisynth_c.h you mentioned that has support for the new colorspaces in 2.6. What sucks is that it is neither source- nor binary-compatible with the avisynth_c.h header from upstream avisynth.
Why don't you create one for upstream avisynth with the baked code removed? I'm sure IanB is interested in it.
Wilbert is offline  
Old 13th October 2013, 21:55   #145  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by Wilbert View Post
Why don't you create one for upstream avisynth with the baked code removed? I'm sure IanB is interested in it.
The largest benefit of that would be so that compatibility to existing C plugins can be kept, like it was done with the 2.6 interface. But that wouldn't solve all the problems. The C interface has multiple structures which were layed out exactly as their C++ counterparts, and they in the end represent core structures, and both Avisynth as well as plugins can depend on their layout. This means it is impossible to modify these types without the possibility of breaking past and future plugins. So even if I removed the baked code, all we'd gain is the ability to modify method definitions. Which is better than nothing I suppose, but I refuse to build a 4th! interface that still has some of the same fundamental problems as the previous three.

That being said, I'm not working on any completely new interface right now. If at all, building a new interface will come later when there is a definite need for it for a feature that is being worked on. But that doesn't mean we cannot think about it until then.

For now, all the new methods that I'm adding to the ScriptEnvironment class are ending up in an "IScriptEnvironment2" abstract class. This has the benefit of never conflicting (layout-wise) with methods that IanB might be inserting into IScriptEnvironment, and it also keeps full compatibilty with existing plugins. Basically, it gives me a nice playground without any risk of breaking plugins until it stabilizes. However, because it is still continuously changing, it is only safe to use IScriptEnvironment2 from inside Avisynth+, or from plugins built and distributed with Avisynth+.

I'll have a real problem though if I ever see the need to modify VideoFrame(Buffer), AVSValue, IClip and so on, thanks to the limitations of the current interfaces. Unlike ScriptEnvironment, these aren't centered around the concept of abstract interfaces like COM-techniques would require.

Last edited by ultim; 13th October 2013 at 21:58.
ultim is offline  
Old 15th October 2013, 22:19   #146  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
I have just updated the online repository and the binaries in the first post with a new version, compiled fresh today. There are some goodies here, so let's see:

- First, Gavino's scripting extensions have been integrated. Be sure to say thanks to Gavino for his work again. I've only made minor modifications to his patches, like properly handling empty "if" blocks or missing optional "else" parts, and taking the "last" variable from before the new "if", "while", and "for" statements better into account. I've also added a "break" statement that will allow you to jump out of any loop without reaching the terminating condition. I've counted the votes for the "for"-style carefully, and your votes turned out equally even, so in the end I picked Gavino's original simplified style. You can still write any kind of sophisticated loop using "while".
- I added back proper AvsPmod support, that got temporarily removed earlier 'coz of the plugin system's rewrite. Built-in functions will now work as before, but to get autoloaded functions to show up in AvsPmod will need a slight modification to AvsPmod. This is unfortunately necessary, because autoloading has to be delayed to support adding search folders in scripts. As soon as AvsPmod gets modified you'll have full functionality back again. The needed modifications won't interfere with traditional Avisynth.
- The C interface is now probed before the 2.5 plugin interface, making ffms2 work again even if you're not using qyot27's latest build.
- The "crop" function now defaults to aligned crop. You can still controll alignment using its second parameter, but if you omit it the default is now for the new frame to be aligned. This is important for plugin authors so that they can have a stronger alignment guarantee, in the end leading to faster processing in multiple plugins.
- And as always, there are cleanups and refactors, in an ongoing effort to make the sources higher quality. Not as many as in previous releases, but still. Of course more to come in the future.

64-bit support
There's actually one more feature item missing from the above list. The archive in the first post includes both 32-bit and 64-bit builds. All ASM from the core got replaced with C-code or intrinsics, the inline ASM in filters got sandwiched in ifdefs, and I fixed up any remaining issues that prevented the core from working correctly. But before rushing to download to run your uberscript in 64-bits, please note that no porting of the builtin filters has been done yet, making that build hardly usable. You'll find that many essential filters are missing, like resizers and color space conversions, just to name a few. So trust me, as a user you are most likely better off using the 32-bit build for now. But the 64-bit is there for the adventurous, for motivated testers and developers, and for all those who are wishing to help port the missing functionality to 64-bit. Besides, the new 64-bit build is compatible with existing 64-bit plugins found here, here and here, so you might actually be able to use the 64-bit version for something if you don't rely much on internal filters.

So, there is a working 64-bit build, though farly gutted out. If you know some intrinsics, please consider helping out with the port, even if with only one or two routines. Even without knowledge of intrinsics or ASM, if you can rewrite some algorithms that were only available in ASM before into plain C, that would already be a lot of help. Feel free to start anywhere you like, and rest assured, I'm also continuing my work on Avisynth+.

Edit: Oh, I almost forgot. I added a small number of issues to the GitHub page. They are just ideas and enhancement requests. If you feel like it, you can also pick up something from there if you prefer working on something new, or you can also add your own ideas.

Last edited by ultim; 15th October 2013 at 23:23.
ultim is offline  
Old 16th October 2013, 01:27   #147  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by ultim View Post
I've also added a "break" statement that will allow ...
thumbs up on that, thanx. EDIT: Indeed, lots of good ideas, GScript & Grunt have been ripe for inclusion for eons.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 16th October 2013 at 01:57.
StainlessS is offline  
Old 16th October 2013, 12:21   #148  |  Link
kypec
User of free A/V tools
 
kypec's Avatar
 
Join Date: Jul 2006
Location: SK
Posts: 826
Missing plugins from 32-bit build?

Hi ultim,

today I've pulled the trigger and wanted to give your build a try but there seems to be something missing or broken in Windows build avsplus_2013_10_15.zip.
I'm using Windows 7 64-bit version and all my scripts were working correctly with AviSynth 2.60 build May 16, 2012.
As soon as I replaced dynamic libraries in my SysWOW64 directory with files from your package (placed under x86 folder) AviSynth.dll and DevIL.dll my script that uses MCTD plugins stopped working, giving this error message:
Code:
AVSMeter 1.7.0 [AVS2.6] by Groucho2004

TemporalSoften::GetFrame is not yet ported to 64-bit.
Here's my script that works fine if MCTD processing is commented out:
Code:
# Initialize variable with path to my A/V tools
plugindir="C:\PROGRAM FILES (x86)\AVTools\plugins\"
# Load all the plugins that will be used in the script
LoadPlugin(plugindir + "DGDecode.dll")
Import(plugindir + "MCTD_Loader.avsi")
# Open source video clip
MPEG2Source("D:\Source\That70sShow\S1E01.d2v", info=3)
# Crop to get rid of tiny overscan and ugly artifacts at the edges
Crop(2, 2, -2, -2)
# Denoise - must be left out to make it work with AviSynth+
MCTemporalDenoise(settings="low", stabilize=true, enhance=true, edgeclean=true, sharp=false)
# Add black borders to get standard mod16 resolution 720x576
AddBorders(2, 2, 2, 2)
Could you please investigate why internal filter TemporalSoften is throwing this misleading error? AFAIK I'm not using 64-bit of AviSynth, I've never had it installed on my system and unless you mixed DLLs in your package I can't see why standard 32-bit should fail on me...

Thanks in advance!
kypec is offline  
Old 16th October 2013, 12:50   #149  |  Link
vdcrim
Registered User
 
Join Date: Dec 2011
Posts: 192
Quote:
Originally Posted by kypec View Post
TemporalSoften::GetFrame is not yet ported to 64-bit
It's a typo.
vdcrim is offline  
Old 16th October 2013, 13:47   #150  |  Link
kypec
User of free A/V tools
 
kypec's Avatar
 
Join Date: Jul 2006
Location: SK
Posts: 826
Quote:
Originally Posted by vdcrim View Post
It's a typo.
Excellent!
Hopefully ultim uploads new fixed build before I even try to get familiar with git on Windows and compile patched sources myself.
kypec is offline  
Old 16th October 2013, 14:59   #151  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
I'm at work, but I'll post a fixed build very soon, I'll be home in a few hours. The bug is only in the most recent build from okt.15. and it mistakenly flags TemporalSoften to be unavailable on 32-bits.
ultim is offline  
Old 16th October 2013, 20:33   #152  |  Link
zero9999
Registered User
 
Join Date: Oct 2011
Posts: 52
(removed build)

Last edited by zero9999; 17th October 2013 at 03:19. Reason: removed build
zero9999 is offline  
Old 16th October 2013, 22:01   #153  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
And we also get an ICC-build Thanks Zero9999!

But for the sake of completeness (or more like coz I've promised) I also updated the release in the first post. It also contains the fix for the typo ofc, and a small number of other things only important for developers (details in the GitHub commit log).

TurboPascal7: I have not forgotten about alignment.h, changes will come next time.
ultim is offline  
Old 16th October 2013, 22:12   #154  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by zero9999 View Post
x86 ICC14 build
That's slower than the VS8 build from ultim. Did you use PGO? Did you read the Intel optimization guides?
Anyway, my experience with Intel compilers is that they get slower with each new version.
Groucho2004 is offline  
Old 16th October 2013, 22:57   #155  |  Link
zero9999
Registered User
 
Join Date: Oct 2011
Posts: 52
Quote:
Originally Posted by Groucho2004 View Post
That's slower than the VS8 build from ultim. Did you use PGO? Did you read the Intel optimization guides?
Anyway, my experience with Intel compilers is that they get slower with each new version.
it was a little faster for the first script i tested with, but using a test script with only the internal functions/filters, it's indeed a little slower. I didn't have much luck with PGO so far, it made the build much slower.

Last edited by zero9999; 16th October 2013 at 23:07.
zero9999 is offline  
Old 17th October 2013, 02:08   #156  |  Link
ARDA
Registered User
 
Join Date: Nov 2001
Posts: 291
@ultim;
First of all thanks for the great effort you are doing.

Quote:
Originally Posted by Groucho2004
That's slower than the VS8 build from ultim. Did you use PGO? Did you read the Intel optimization guides?
Anyway, my experience with Intel compilers is that they get slower with each new version.
Quote:
Originally Posted by zero9999
it was a little faster for the first script i tested with, but using a test script with only the
internal functions/filters, it's indeed a little slower. I didn't have much luck with PGO so far,
it made the build much slower.
I don't know if this is the right thread, I have no intention to pollute it.
But as we are going to have several and different compiled versions of avisynth around, I would like to arrive
to a common benchmark tool and conditions, so we can compare.

@Groucho2004 and zero9999;
Will you please post your benchmark methods with most possible details ?

Related to Gpo or cpu dispatch options in compilers, I had no good experience with avisynth
in that matter. I think the question is that in many in built functions in avisynth mmx, isse
sse2, sse3 and ssse3 inline asm code cannot be touched by the compiler, so just c++ will be
optimized or cpu dispatched which obviously will improve almost nothing for that cases.
Some years ago I tried with intel compiler 10 but never could get any good result, but remember
also (don't know about newer versions) that in some cases when those codes run in a AMD
cpu were derivated to a general not optimized library.

Thanks ARDA
ARDA is offline  
Old 17th October 2013, 02:19   #157  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
There is not much point in compiling avs+ with ICL right now. However it will be "a must" for optimal performance after some code gets ported to intrinsics as VC tends to do a pretty bad job here. I also can confirm that auto-vectorization does not and probably will never help avs core much.

Next, I do not believe that there should be several differently compiled version of avisynth flying around and I would like to ask other people to refrain from posting binaries at least for now as it might just create additional confusion. In the end I believe the best solution would be to provide one binary, built with ICL, as code it generates for intrinsics seems to be faster than VC even on AMD processors. Getting ICL is not hard even if you are not a student.

As for benchmark methods, I think AvsMeter is pretty much the standard tool right now.

Last edited by TurboPascal7; 17th October 2013 at 02:35.
TurboPascal7 is offline  
Old 17th October 2013, 02:35   #158  |  Link
ARDA
Registered User
 
Join Date: Nov 2001
Posts: 291
Quote:
Originally Posted by TurboPascal7
Next, I do not believe that there should be several differently compiled version of avisynth flying around
I would like to ask other people to refrain from posting binaries at least for now as it might just create
additional confusion. In the end I believe the best solution would be to provide one binary,
built with ICL, as code it generates for intrinsics seems to be faster than VC even on AMD processors.
Getting ICL is not hard even if you are not a student.
I second

Quote:
Originally Posted by ARDA
...but remember also (don't know about newer versions) that in some cases when those codes run in a AMD
cpu were derivated to a general not optimized library.
I was talking about this http://www.agner.org/optimize/blog/read.php?i=49#49

Thanks
ARDA is offline  
Old 17th October 2013, 02:38   #159  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
This is not relevant here because avisynth doesn't use any Intel libraries and will not rely on automatic vectorization in the future. Better register allocation and instruction ordering that ICL offers for intrinsics is still better even on AMD processors (but probably not optimal).
TurboPascal7 is offline  
Old 17th October 2013, 02:48   #160  |  Link
ARDA
Registered User
 
Join Date: Nov 2001
Posts: 291
Quote:
Originally Posted by TurboPascal7
This is not relevant here because avisynth doesn't use any Intel libraries
I don't understand. When you compile with intel compilers, just to mention a few by heart it has it own
memcpy, memset, memmcmp, all strings libraries, etc, etc, etc.
I don't know if that legal question was solved and it is not present in nowadays versions.
I don't want to pollute this thread, so for me this subject is enough discussed.

I'll be waiting Groucho2004 answer about benchmark methods,

Thanks

Last edited by ARDA; 17th October 2013 at 02:53.
ARDA is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.