Avisynth+

[Groucho2004]

Help. Windows 10 is too smart, and is trying to protect me from myself :)
I have troubles to put properly avisynth.dll into c:\windows\system32.
Once I had there an r1825 version, then I copied the r1828 over it.
Avsmeter64 is still showing avisynth+ version r1825.
Then I purged avisynth.dll from c:\windows\system32.
And a (not so big) surprise: avs scripts are still running fine using a ghost r1825. Where is this dll cache? How can I tell windows 10 to clear all historical use of avisynth.dll?
"AVSMeter64 -avsinfo" shows you the path to the avisynth.dll it loads. What does it show?

[pinterf]

"AVSMeter64 -avsinfo" shows you the path to the avisynth.dll it loads. What does it show?
Machine is not stupid. Lazy non-commandline man is.
I made the copy from a 32 bit Total Commander, that shows c:\windows\SysWOW64 content under c:\windows\system32. And I always worked into that. Grrr.

[Groucho2004]

Machine is not stupid. Lazy non-commandline man is.
I made the copy from a 32 bit Total Commander, that shows c:\windows\SysWOW64 content under c:\windows\system32. And I always worked into that. Grrr.
Yeah, I had that problem before. The only reliable way to get the correct system32/syswow64 directories across all Windows versions is to go through "CreateFileMapping()", "MapViewOfFile()", "GetLogicalDriveStrings()", "QueryDosDevice()".
Have a look at the "GetFileNameFromHandle()" function in AVSMeter's source (AvisynthInfo.h).

I didn't expect Total Commander would fall into that trap.

[manolito]

That's one of the most important reasons why I still love WinXP. I just hate it to no end when the operating system lies to me about the true location of my files.

[Groucho2004]

That's one of the most important reasons why I still love WinXP. I just hate it to no end when the operating system lies to me about the true location of my files.
The GetModuleFileName() WoW64 redirection issue is also present in XP64 and Server 2003 x64. Apparently, it was fixed in Win7 and re-appeared in Win8 (and, it would seem, Win10).

Some info in the comment section here (https://technet.microsoft.com/en-us/library/ms683197.aspx).

[thescrapyard]

Help. Windows 10 is too smart, and is trying to protect me from myself :)
I have troubles to put properly avisynth.dll into c:\windows\system32.
Once I had there an r1825 version, then I copied the r1828 over it.
Avsmeter64 is still showing avisynth+ version r1825.
Then I purged avisynth.dll from c:\windows\system32.
And a (not so big) surprise: avs scripts are still running fine using a ghost r1825. Where is this dll cache? How can I tell windows 10 to clear all historical use of avisynth.dll?


Disable UAC, no nagging prompts about allowing things to run everytime, even if software runs correctly it will keep poping up until it 'learns' whats safe and isn't. Turn it OFF and everything is considered safe. Also block software calling other software, such as AviSynth calling plugins etc etc from non-standard file locations outside windows 10 control


http://winaero.com/blog/how-to-turn-off-and-disable-uac-in-windows-10/

Your virus software should protect you from any nasties


I've just sorted my daughters brand new laptop that came with Win10. First thing I did was disable UAC and installed decent security suite and Office so she's happy now

[Groucho2004]

Disable UAC
:confused: What does this have to do with the redirection issue?

[AzraelNewtype]

Nothing, it's just some stunningly bad advice.

[TheFluff]

Yeah, I had that problem before. The only reliable way to get the correct system32/syswow64 directories across all Windows versions is to go through "CreateFileMapping()", "MapViewOfFile()", "GetLogicalDriveStrings()", "QueryDosDevice()".
Have a look at the "GetFileNameFromHandle()" function in AVSMeter's source (AvisynthInfo.h).

I didn't expect Total Commander would fall into that trap.

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365743(v=vs.85).aspx

hope this helps

Help. Windows 10 is too smart, and is trying to protect me from myself :)
I have troubles to put properly avisynth.dll into c:\windows\system32.
Once I had there an r1825 version, then I copied the r1828 over it.
Avsmeter64 is still showing avisynth+ version r1825.
Then I purged avisynth.dll from c:\windows\system32.
And a (not so big) surprise: avs scripts are still running fine using a ghost r1825. Where is this dll cache? How can I tell windows 10 to clear all historical use of avisynth.dll?

It's 2016, stop dumping random dll's into system folders. There's absolutely no reason for you to be mucking around in there. Then again it's not really your fault, Avisynth should have stopped putting the dll in there in the first place like four major Windows versions ago.

[Groucho2004]

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365743(v=vs.85).aspx

hope this helps
Been there, done that. I have had the same frustrating experience with "Wow64DisableWow64FsRedirection()" as the user who reported in the comment section of "GetModuleFileName()" on technet. It just does not work, at least not for every OS.

[TheFluff]

Been there, done that. I have had the same frustrating experience with "Wow64DisableWow64FsRedirection()" as the user who reported in the comment section of "GetModuleFileName()" on technet. It just does not work, at least not for every OS.

wontfix: working as intended. if you use xp64 you deserve to suffer.

[Groucho2004]

if you use xp64 you deserve to suffer.I didn't expect any other reply from you. :)

[Groucho2004]

It's 2016, stop dumping random dll's into system folders. There's absolutely no reason for you to be mucking around in there. Then again it's not really your fault, Avisynth should have stopped putting the dll in there in the first place like four major Windows versions ago.
Where else would you put it?

Let's explore the options:
1. System32/SysWoW64
Every client can find and load avisynth.dll, be it 32 or 64 bit. 32 and 64 bit Avisynth can both be installed.

2. Any directory that is referenced in the "PATH" environment variable.
The installer would need to parse the PATH environment and then ask the user where it should be placed. Also, this only works for either 32 or 64 bit.

You decide which one is better.

[TheFluff]

It doesn't need to be on the $PATH (actually, it shouldn't be on it). It's kinda late to do anything about all the ancient third-party software now, but the Right Thing is to just install into whatever folder the user chooses and use a registry key (different ones for 32- and 64-bit, obvs) to point to its location. VS does this. VfW programs can still load it as normal and everything that needs to interface with the DLL directly can find it via the registry key (or by letting the user point it out, if you want a portable version). Installing it globally was acceptable when Avisynth was designed (and you kinda had to do it, AFAIK?) but it's an all around awful idea. It's one of the many things Avs+ should've broken backwards compatibility with when they had the chance.

2. Any directory that is referenced in the "PATH" environment variable.
The installer would need to parse the PATH environment and then ask the user where it should be placed. Also, this only works for either 32 or 64 bit.
I haven't tried it, but the internet claims that if you have a 32-bit lib in one dir and the same lib except 64-bit in another dir, and both those dirs are on the $PATH, LoadLibrary is smart enough to figure out which one to grab for your process. Not that it's a thing you should do.

[Groucho2004]

It doesn't need to be on the $PATH (and probably definitely shouldn't be). It's kinda late to do anything about all the ancient third-party software now, but the Right Thing is to just install into whatever folder the user chooses and use a registry key (different ones for 32- and 64-bit, obvs) to point to its location. VS does this. VfW programs can still load it as normal and everything that needs to interface with the DLL directly can find it via the registry key (or by letting the user point it out, if you want a portable version). Installing it globally was acceptable when Avisynth was designed (and you kinda had to do it, AFAIK?) but it's an all around awful idea. It's one of the many things Avs+ should've broken backwards compatibility with when they had the chance.
Not sure what you mean by "ancient third party software". Pretty much every client program that uses Avisynth loads avisynth.dll through "LoadLibrary()" implicitly, i.e. without specifying a fully qualified path. If avisynth.dll is not in one of the directories included in the standard Windows search hierarchy, it fails to load.
Therefore, the code of each of these clients (VDub, AVSP, x264, mpc(hc), ...) would have to be changed.

And yes, in hindsight it may have been a good idea to use registry pointers.

[thescrapyard]

:confused: What does this have to do with the redirection issue?

Just trying to be slightly helpful and assumed the problem was windows deciding it knew best and thought it was blocking the DLL being put where he wanted it to be and the software was looking for it, as has happened to me when the OS thought everything was a suspicious file so either silently blocked it or poppingup nag requesters asking if its OK to run when installed software that calls other software

Such as AviSynth or other ones I use such as AVStoDVD that also had issues in the past. Not with the package but UAC blocking everything from running and permissions aggravation

[edcrfv94]

Scripts add to ffdshow with r1828-pfmod have no effect.

[LigH]

I have no clue, but wonder if ffdshow possibly supports only an outdated API version and may not be compatible with AviSynth+ regarding internal post-processing scripts?

[chainik_svp]

That commit (https://github.com/AviSynth/AviSynthPlus/commit/ec72100460f7ad5eb7ed22c599a20fe44a154805) broke ffdshow.
It's impossible now for ffdshow to add a function into ScriptEnv.

PluginManager::AddFunction()
...
newFunc = AVSFunction(name, NULL, params, apply, user_data);
assert(newFunc.IsScriptFunction());

...

AVSFunction::AVSFunction(const char* _name, const char* _plugin_basename, ...)
...
if ((NULL != _name) && (NULL != _plugin_basename))

[jpsdr]

One note, my mod was built with VS2015, v140_xp toolset.
I have no "virgin" machine, so it works out-of-box for me. It may require Visual C++ Redistributable for Visual Studio 2015 Update 1 (https://www.microsoft.com/en-us/download/details.aspx?id=49984) for you.

Unfortunately, XP support is partialy broken in VS2015, and it's still the case with Update 2.
For having a proper support of XP, as it's explained here (https://connect.microsoft.com/VisualStudio/feedback/details/1789709/visual-c-2015-runtime-broken-on-windows-server-2003-c-11-magic-statics), you have to compile with the /Zc:threadSafeInit- option, otherwise it will not work.
My advice would be, if you want to support XP, to make 2 builds.
One generic with v140 toolset and no XP support, and another XP support specific, with v140_xp toolset and the /Zc:threadSafeInit- option.

[bxyhxyh]

Help. Windows 10 is too smart, and is trying to protect me from myself
I have troubles to put properly avisynth.dll into c:\windows\system32.
Once I had there an r1825 version, then I copied the r1828 over it.
Avsmeter64 is still showing avisynth+ version r1825.
Then I purged avisynth.dll from c:\windows\system32.
And a (not so big) surprise: avs scripts are still running fine using a ghost r1825. Where is this dll cache? How can I tell windows 10 to clear all historical use of avisynth.dll?

When I was trying to use Groucho's avisynth switching tool, I had similar issue (http://forum.doom9.org/showthread.php?p=1716779#post1716779)
For your case, isn't it in syswow64 folder?
Anyway If it's 64bit Windows, shouldn't you replace syswow64's dll?
I don't know if it would help you since you guys are professional programmers.

[asarian]

When I was trying to use Groucho's avisynth switching tool, I had similar issue (http://forum.doom9.org/showthread.php?p=1716779#post1716779)
For your case, isn't it in syswow64 folder?
Anyway If it's 64bit Windows, shouldn't you replace syswow64's dll?
I don't know if it would help you since you guys are professional programmers.

The 32-bit AviSynth DLL goes into the syswow64 folder (and vice versa). Don't ask me why, but that's how it works. :)

[pinterf]

When I was trying to use Groucho's avisynth switching tool, I had similar issue (http://forum.doom9.org/showthread.php?p=1716779#post1716779)
For your case, isn't it in syswow64 folder?
Anyway If it's 64bit Windows, shouldn't you replace syswow64's dll?
I don't know if it would help you since you guys are professional programmers.
The solution was using 64bit Total Commander because 32 bit version was showing the syswow64 folder content while pretending that i am in system32.

Tapatalkkal küldve az én GT-I9195I eszközömről

[Groucho2004]

The 32-bit AviSynth DLL goes into the syswow64 folder (and vice versa). Don't ask me why, but that's how it works. :)
64 Bit Windows versions have a 32 bit sub-system called WoW64. It stands for "Windows on Windows64" which makes it possible to run 32 bit processes/applications on 64 bit Windows. I suppose it would be less ambiguous if it was called "W32oW64" or similar. So, "SysWoW64" is the system directory for 32 bit DLLs called by 32 bit processes on 64 bit Windows.

The issue that pinterf had was rectified, it had to do with inconsistent behaviour of file system redirection functions in the Windows API across various Windows versions.

[pinterf]

Unfortunately, XP support is partialy broken in VS2015, and it's still the case with Update 2.
For having a proper support of XP, as it's explained here (https://connect.microsoft.com/VisualStudio/feedback/details/1789709/visual-c-2015-runtime-broken-on-windows-server-2003-c-11-magic-statics), you have to compile with the /Zc:threadSafeInit- option, otherwise it will not work.
My advice would be, if you want to support XP, to make 2 builds.
One generic with v140 toolset and no XP support, and another XP support specific, with v140_xp toolset and the /Zc:threadSafeInit- option.
Thank you, it works! Without the option (with the "-" at the end, first I missed it) I get a fine 0xC0000005.
(I realized that I have a Virtual XP in my Win7Prof).
I will make a build later, since I made more debug mess in the code than you can imagine.

Some progress:
Last week I tried to narrow down the random-increasing memory consumption problem that I wrote earlier about. Frankly I was getting mad. The problem occured only when passing the 35800th frame and also from that time on in random waves. It's not easy to debug something that first appears after 15-20 minutes of running. This problem does not occur with colorbars, nor with a modded (non-static base frame) colorbars.
Only with a DV-AVI. I already debugged the cache system (nice one :-)), put timestamps to each framebuffer registration to see which frames got stuck and did not get reused in the last 10 seconds.

Yesterday I happily realized that it happens both with avisource or ffms2 so I could exclude them from the culprit list.

After one week I found out that the behaviour is not time-dependent, today I made a brave Trim() :) from before the problematic frames and wonder! it started to eat memory from the same source frames. All I know that these frames have heavy camera shake.

So it seems that one of the plugins in qtgmc(Fast) is leaking frames under these speficic circumstances.

At least now I don't have to wait forever for the problem to occur.

[LigH]

Tested with AviSynth+ r1825 MT and r1828-pfmod (MSVC2015 Redist installed):

LoadPlugin("E:\Programme\AviSynth 2.5\plugins\LSMASHSource\LSMASHSource.dll")
a=LwLibavAudioSource("truehd.mka").AssumeFPS(50,1)
v=ColorBars(1280,720,"YV12").AssumeFPS(50,1).Trim(0,Round(a.AudioDuration*50.0))
AudioDub(v,a)
Info()
Histogram(mode="Audiolevels")

crashes upon loading; AVSMeter (and VirtualDubMod similarly) report:

AVSMeter 2.1.6 (x86)
AviSynth+ 0.1 (r1828, MT-pfmod, i386) (0.1.0.0)

System exception - Integer Divide by Zero
(H:\Video\Test\truehd.avs, line 6)

It works when I comment out the Histogram() command in line #6.

In original AviSynth up to v2.6.0 MT final, it loads the TrueHD track with 8 channels and displays audio levels overlayed.

[Reel.Deel]

System exception - Integer Divide by Zero


raffriff42 posted similar problems a while back also. Take a look at post #1251 through #1253 (http://forum.doom9.org/showthread.php?p=1737382#post1737382).


@pinterf
If is not a big deal, can you include PR #64 (https://github.com/AviSynth/AviSynthPlus/pull/64) with your next build?

[Chikuzen]

https://github.com/AviSynth/AviSynthPlus/pull/65/commits/9b1745694e6d527cad04d8a8f3625dde2a736b87

https://dl.dropboxusercontent.com/u/19797864/forum/fix_audio.jpg

[pinterf]

Ok. I will check it.

[pinterf]

Avisynth+ r1841-pfmod

http://www.mediafire.com/download/eadynn3jpv720jl/avsplus-r1841-pfmod.7z

From readme:

AvisynthPlus r1841-pfmod
2016.04.08
- TemporalSoften frame leak fix (short to write but it took me 30 hours :) )
- minor FrameRegistry2 tunings
- PR #64 rgb24<->rgb32 ssse mod: https://github.com/AviSynth/AviSynthPlus/pull/64
- PR #65 Fix audio cache #65 (Chikuzen)
- Colorbars new parameter: Boolean staticframes
true (default): returns the precalculated static frame's pointer
false: copies the precalculated frame into a new frame and returnes this new frame pointer
- XP versions: v140_xp toolset with extra C++ commandline option /Zc:threadSafeInit-

Very good memory consumption
r1841:
QTGMC(Fast) on x64 + Prefetch(8) + 720x576 DV-AVI Source = 350-400 MB

r1828:
QTGMC(Fast) on x64 + no MT + 720x576 DV-AVI Source = 200->1900 MB increasing

Please test it, good night!

[TheFluff]

7z files on mediafire isn't a reasonable way to contribute to open source projects in 2016. Please consider putting your stuff on github so other people can easily backport your changes to the official repo or to their own forks if they want.

e: oh wait your source isn't even in the 7z. Not that anyone cares, but you should really put it up somewhere or you're technically violating the GPL.

[Reel.Deel]

Avisynth+ r1841-pfmod

http://www.mediafire.com/download/eadynn3jpv720jl/avsplus-r1841-pfmod.7z


Thanks pinterf! Will definitely test it out.


7z files on mediafire isn't a reasonable way to contribute to open source projects in 2016. Please consider putting your stuff on github so other people can easily backport your changes to the official repo or to their own forks if they want.

e: oh wait your source isn't even in the 7z. Not that anyone cares, but you should really put it up somewhere or you're technically violating the GPL.

It's already on GitHub: https://github.com/pinterf/AviSynthPlus/tree/MT-pfmod

[raffriff42]

It's fixed my little complaint, so I can come back to AVS+. Thanks pinterf!

[TheFluff]

It's already on GitHub: https://github.com/pinterf/AviSynthPlus/tree/MT-pfmod
Oh okay, sorry. I didn't see it in the fork list for some reason.

[chainik_svp]

ffdshow fix: https://github.com/pinterf/AviSynthPlus/pull/1

sadly memory leak when the script environment is re-inited by the ffdshow 64-bit still not fixed... (while it can be a ffdshow's issue)

[Groucho2004]

sadly memory leak when the script environment is re-inited by the ffdshow 64-bit still not fixed... (while it can be a ffdshow's issue)
I'd say it's definitely a ffdshow issue. ffdshow initialises IScriptEnvironment in several places and releases it with "delete env;" and "delete clip;". It's recipe for disaster. With AVS 2.6 one can use "DeleteScriptEnvironment()" which properly releases the pointers and memory.

Are you sure the same does not happen with 32 bit ffdshow?

[chainik_svp]

> Are you sure the same does not happen with 32 bit ffdshow?

Absolutely. It's like hundreds of MBs on each re-init.
We've already talked about this back in March 2015 as I can remember... :)

[Groucho2004]

We've already talked about this back in March 2015 as I can remember... :)
True, just found it.

[jones1913]

Avisynth+ r1841-pfmod

http://www.mediafire.com/download/eadynn3jpv720jl/avsplus-r1841-pfmod.7z

Thanks, this has also fixed the crashes I had with QTGMC and AVS+.

[pinterf]

Now I have a special 32 bit debug build that only differs from the working version that there is a 1/10000 or 1/1000 sec wait cycle in the caching code in the "not found but get frame from child then push it in cache".
This delay results in zero pointer frames appearing in the cache. My tests run fine with mt mode 2 but with this version Qtgmc fast is giving C0000005 exceptions at random places, if thread count is big enough. E.g. prefetch(8).
If I set all masktools2 filters to mt mode 1 (serialize) then it works again.
But its not because of masktools. Having its filters serialized only modifies the timing conditions.
There must be a race condition, the lru cacheing code seems to be perfect, I suspect the prefetcher and the worker threads and the caches are not perfercly synchronized.
Then it turned out that I can make it to freeze with simple 3 lines. When i will back at my PC I will show you.

Is there anybody here who was deeply involved in these mt and prefetch queue and cache core things? Not an easy read to reverse engineer it :)

[chainik_svp]

I can try! :D

[Chikuzen]

Is there anybody here who was deeply involved in these mt and prefetch queue and cache core things? Not an easy read to reverse engineer it :)

I think the man is tp7.
I recommend you to just send a mail to him.

[TurboPascal7]

tp7 hardly remembers anything about the threading codebase, pretty much all of which was written by ultim.
Have fun.

[pinterf]

Thanks tp7, anyway.
There are clear drawbacks of a one man show. At least the codebase is pretty nice.

[pinterf]

Meanwhile I think the cache corruption is solved.

Problem:
- a debug build with an extra command placed in the caching code
- random crashes 0xC0000005, null frame pointers, etc..., somewhere in the big black hole of QTGMC
- could not debug in IDE, since the 8 threads and 1/10000 sec timing differences made the problem disappear. Debugging was possible only by analyzing long debug outputs.

After some days I managed to shrink down the original QTGMC.avsi to this very-very complex script:
Blankclip(width = 320, height = 200, pixel_type = "yv12").KillAudio()
# no freeze without KillAudio. KillAudio is giving us a new caching level
SeparateFields()
Prefetch(8)
Nice, isn't is?

And this is the code fragment that deals with case, when the Nth frame is not found in cache. (Sorry, just to feel the pain :) )
PVideoFrame result;
LruCache<size_t, PVideoFrame>::handle cache_handle;

switch(_pimpl->VideoCache->lookup(n, &cache_handle, true))
{
case LRU_LOOKUP_NOT_FOUND:
{
try
{
cache_handle.first->value = _pimpl->child->GetFrame(n, env);
#ifdef X86_32
_mm_empty();
#endif
_pimpl->VideoCache->commit_value(&cache_handle);
}
catch (...)
{
_pimpl->VideoCache->rollback(&cache_handle);
throw;
}
#ifdef _DEBUG
// !!! some process during the next 1/10000 seconds is overwriting the
// content of this cache handle (frame) with NULL!
// 1/10000 sec delay, but a simple _RPT debug line is enough, albeit the
// corruption occurs more rarely
std::chrono::time_point<std::chrono::high_resolution_clock> t_start2, t_end2;
std::chrono::duration<double> elapsed_seconds;
t_start2 = std::chrono::high_resolution_clock::now();
do {
t_end2 = std::chrono::high_resolution_clock::now();
elapsed_seconds = t_end2 - t_start2;
} while (elapsed_seconds.count() < 1.0 / 10000.0);
// end of delay
assert(NULL != cache_handle.first->value); // and now it's NULL!!!
#endif
result = cache_handle.first->value;
// its content may change after commit when the last lock is released
// (cache is being restructured by other threads, new frames?)
break;
[...]


Solution: fill 'result' _before_ the commit. Easy eh? Six letters and plus one line and a deleted line at the bottom. It took me three or four days, again. But now you can ask me about the caching.

Although it happened only to the debug version, theoretically it would occur anytime. I don't know, at different processor speeds, thread count, load, etc...

And as a new adventure I had to plunge into masktools2 a bit when I thought it would be the suspect, since putting its filters into serialized MT mode, the corruption did not occur in QTGMC. It was false alarm, it works fine in mode 2 (MT_MULTI_INSTANCE).

[ryrynz]

Meanwhile I think the cache corruption is solved.


Great work, I think this entire thread now loves you. All hail the new Ultim.

[LigH]

Let's be happy about it:

http://cosgan.de/images/smilie/musik/e050.gif http://cosgan.de/images/smilie/musik/n015.gif

[pinterf]

Haha :)
Then all readers here are obeyed to run with me on that 54K 2800D+
Just for plain solidarity :)

[pinterf]

(Those numbers are not the newest avs+ versioning scheme but the data of the trail running race next weekend on which I trained almost nothing because of these avs+ riddles. :)

[pinterf]

New AVS+ build: r1847

x64/x86, including XP versions

r1847 MT-pfmod (2016.04.18)
- fixed broken ffdshow integration (chainikdn)
- fixed theoretical and debug case cache corruption

Download from
https://github.com/pinterf/AviSynthPlus/releases/tag/r1847-MT-pfmod
or
http://www.mediafire.com/download/r7bgaq94cbr9kmf/avsplus-r1847-pfmod.7z