AviSynth+ migration

[spoRv]

New year, new tasks, so I'm thinking to migrate to AviSynth+ (from 2.60), but I'm a bit afraid...

I'll do this primarily for memory problems I encounter sometimes - where + should have not; if the speed would improve, it will be great as well!

The main question is: do all the normal AviSynth plugins work with AviSynth+? I mean, it is 100% compatible, in theory, but in practice? Are there some well known exceptions?

Thanks to everyone!

[Groucho2004]

The main question is: do all the normal AviSynth plugins work with AviSynth+? I mean, it is 100% compatible, in theory, but in practice? Are there some well known exceptions?They should all work but there are indeed a couple of exceptions - The rather ancient CPP 2.0 and C2.0 plugins do not work with AVS+.

Examples:

C2.0 plugins:
AVSCurveFlow.dll
AVSShock.dll
IBob.dll
Invert_Pascal.dll

CPP 2.0 plugins:
DustV5.dll

[StainlessS]

At least one of RT_Stats functions do not behave the same.
I think the below may not work as expected due to the internal API function returning an empty string "".

RT_PluginDir # pretty sure dont work (""), as multiple plugins directories


Dont recall any others, If additional then perhaps Pinterf could add to list (of internal API funcs that return "" rather than avs2.6 standad result).

Not sure about these

RT_PluginFunctions
RT_PluginParam


EDIT:
Also, I'm pretty sure that PluginParam/RT_PluginParam returns only a single parameter list for both avs+ and standard, where mutliple alternative arguments lists are provided for the same function.
(The API function only returns a single arument list [maybe the last, or maybe the first, dont recall]).

eg multiple argument/parameter list [[B]From docs only, not necessarily as specified in source code]

AssumeFPS (clip, float fps, bool "sync_audio")
AssumeFPS (clip, int numerator [, int denominator], bool "sync_audio")
AssumeFPS (clip1, clip2, bool "sync_audio") <<<-------- Below Edit returns this one
AssumeFPS (clip, string preset, bool "sync_audio")

Above is a limitation in API, so is same in avs+ or standard.
EDIT:
Maybe should take an optional int eg PluginParam(String FuncName,int "index"=0) which returns "" when no additional parameter lists.
Perhaps same type thing (or variation) for PluginDir.

EDIT: Source code

{ "AssumeFPS", "ci[]i[sync_audio]b", AssumeFPS::Create }, // dst framerate, sync audio?
{ "AssumeFPS", "cf[sync_audio]b", AssumeFPS::CreateFloat }, // dst framerate, sync audio?
{ "AssumeFPS", "cs[sync_audio]b", AssumeFPS::CreatePreset }, // dst framerate, sync audio?
{ "AssumeFPS", "cc[sync_audio]b", AssumeFPS::CreateFromClip }, // clip with dst framerate, sync audio?

For Assumefps, returns this one (last one specified in source)

AssumeFPS "cc[sync_audio]b"


EDIT: Some plugins may have dropped support for YUY2, instead supporting YV16, (dont have a list of them).

[spoRv]

Thanks guys!

So, we can say that it's pretty sure that around 99% of the plugins/filters should work, right?
Also, would they work with AviSynth+ 64bit? And, shouldn't they be a bit faster than using AviSynth 2.60 32bit?

At the end, would it be unharmful/unpainful for a dinosaur like me to migrate to AviSynth+? ;)

[StainlessS]

So, we can say that it's pretty sure that around 99% of the plugins/filters should work, right?

About right, but some scripts that rely on YUY2 might break.
EDIT: Above line really relates to plugins that have been modified for avs+ colorspaces, and mod coders were a bit lazy or
awkward in removing YUY2 support, resulting in breaking some scripts.


Also, would they work with AviSynth+ 64bit? And, shouldn't they be a bit faster than using AviSynth 2.60 32bit?

Only 64bit converted plugs work in AVS+ 64 bit.
Any general purpose 32bit -> 64bit Exe conversion is likely to be about 5% to 15% slower (due to double size of pointers and also perhaps array indexes), though video plugs may be faster as more ram can be used for cache.
EDIT: With 4GB or less ram, I cannot think of any upside to 64bit, unless for eg testing 64bit software (including OS).
Although, I'm not sure of effect of using 32bit as opposed to 64 bit on a 64 Bit system. (above is really comparing 32 bit on 32 bit system and 64 on 64 bit system)
EDIT: Above in red is incorrect.


At the end, would it be unharmful/unpainful for a dinosaur like me to migrate to AviSynth+? ;)

I hope so, I'm considering doing soon too, let us all know how it goes.

[raffriff42]

Suggest getting Groucho2005's Universal Installer.
http://forum.doom9.org/showthread.php?t=172124

This lets you revert quickly in case of a problem (of which there will be few, and none that can't be easily worked around)

EDIT start with 32-bit.

[Groucho2004]

Groucho2005He's still in training but will take over business once I retire. :D

[spoRv]

Well, I think I'll start with the 32bit, though - but, do this have memory limited as normal AviSynth?

He's still in training but will take over business once I retire. :D

LOL!!!

[Groucho2004]

Well, I think I'll start with the 32bit, though - but, do this have memory limited as normal AviSynth?AVS+ generally uses less memory than the official Avisynth, at least in my experience. Of course, memory usage also depends on the plugins/scripts used.

You can use AVSMeter to monitor memory usage.

Edit: The usual 32 bit memory limit (2 Gigs) is the same with all 32 bit Avisynth versions. On a 64 bit OS this can be expanded to 4 Gigs which should be enough for most scenarios.

[pinterf]

Any general purpose 32bit -> 64bit Exe conversion is likely to be about 5% to 15% slower (due to double size of pointers and also perhaps array indexes), though video plugs may be faster as more ram can be used for cache.

I think the slowdown due to double pointer sizes is neglible. In x64 there are much more general purpose and XMM/YMM registers are available, compiled code can be more efficient because processor registers are used instead of temporary variables swapped from/to memory.

[StainlessS]

I think I read the figures 5% -> 15% on the intel linked site:- https://software.intel.com/en-us/articles/lessons-on-development-of-64-bit-cc-applications/
(Or maybe it was on StackExchange).

My only experience coding for 64 bit did result in slower 64 bit, but may have been down to ME :(
Here (WaterMark):- https://forum.doom9.org/showthread.php?p=1826340#post1826340

EDIT: Although it did make it more awkward in my 64bit version as I changed from using RGB blue channel of mask image for the mask (which I did not like) to the YUV luma channel, and in doing so made it necessary to track the mask in YUV luma and apply it to RGB32 R,G,B channels, and so having Y the right way up, and RGB upside down.

[Groucho2004]

I found that in many cases 64 bit uses slightly less memory.

For example:
Script:
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio().assumefps(50, 1).trim(0, 499)

RemoveNoise()

function RemoveNoise(clip video, int "threshold")
{
last = video
threshold = default(threshold, 300)

sc = MSuper(hpad = 16, vpad = 16)
backward_vector = MAnalyse(sc, isb = true, delta = 1, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
forward_vector = MAnalyse(sc, isb = false, delta = 1, blksize = 16, overlap = 4, truemotion = false, sadx264 = 4)
MDegrain1(sc, backward_vector, forward_vector, thSAD = threshold)

return last
}


32 bit:
Frames processed: 500 (0 - 499)
FPS (min | max | average): 17.38 | 72.00 | 30.01
Memory usage (phys | virt): 102 | 98 MiB
Thread count: 9
CPU usage (average): 25%

64 bit
Frames processed: 500 (0 - 499)
FPS (min | max | average): 17.76 | 77.57 | 31.38
Memory usage (phys | virt): 95 | 91 MiB
Thread count: 9
CPU usage (average): 25%

[StainlessS]

Oops sorry, got it back-to-front.
From the Intel site, link in my prev post. about (5-10 percent down the page, its a big page)

Compilation of 64-bit code does not only allow you to avoid using WoW64 but also gives you an additional performance gain.
This is explained by architectural modifications in the microprocessor such as an increased number of general-purpose registers.
For an average program, you may expect a 5-15% performance gain after mere recompilation.

EDIT: And a bit more

Benefits of the 64-bit environment for 32-bit programs

Because of the WoW64 layer, 32-bit programs are less efficient in the 64-bit environment than in their native 32-bit one.
But still simple 32-bit applications can get one benefit of being executed in the 64-bit environment. #
Maybe you know that a program built with the switch "/LARGEADDRESSAWARE:YES" can allocate up to 3 Gbytes of memory if a 32-bit
Windows is launched with the switch "/3gb". Well, the same 32-bit program built on a 64-bit system can allocate almost 4 Gbytes of memory
(in practice it is usually about 3.5 Gbytes).

EDIT: And the link again:- https://software.intel.com/en-us/articles/lessons-on-development-of-64-bit-cc-applications/

[Bexley]

I did my annual reformat/reinstall of Windows this week, and I decided to also make the move to Avisynth+ while I was at it. I've played around with it a little in the past but never felt like it was mature enough to make the switch, but I looked at the current ported plugin list and decided to take the plunge. Using r2580-MT

Having some issues, though. I'm finding it rather unstable compared to 2.6MT. I'm converting my DVD collection to x264 for use in my media server, so I'm ripping DVDs and serving from AVS to MeGUI. MT performance seems to be around 20% slower than 2.6 no matter how many prefetch threads I use, and I get random hangs during encoding that I never used to get. I've also seen some encodes just abort for no apparent reason in the middle. I never had any of these issues before with 2.6, and some of this (particularly MT speed) was the reason I never stayed with it before. On some shows I use no filtering and have no issues, on others (old ones) I use a combination of TemporalDegrain and RemoveDirtMC

Typical 2.6 script where I encode in real time.

SetMTMode(3)
DGDecode_mpeg2source("C:\ADVENTURES_OF_SUPERMAN_DISC_3\VIDEO_TS\VTS_01_1.d2v", info=3)
SetMTMode(2)
TemporalDegrain(degrain=1,ov=2,hq=1)
RemoveDirtMC(25)
Greyscale()


For AVS+ I use the MT mode list here (http://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest) and autoload it as mtmodes.avsi. I get 18-20fps with this script. (it's a 6 core CPU and I've tried everything from 4 to 12 threads)

DGDecode_mpeg2source("C:\ADVENTURES_OF_SUPERMAN_DISC_3\VIDEO_TS\VTS_01_1.d2v", info=3)
TemporalDegrain(degrain=1,ov=2,hq=1)
RemoveDirtMC(25)
Greyscale()
Prefetch(8)

About 1 in 4 encodes will actually complete without hanging, which makes it essentially useless for me. Is this just the state of AVS+, or maybe a compatibility issue with plugins? Any ideas?

[Groucho2004]

For AVS+ I use the MT mode list here (http://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest) and autoload it as mtmodes.avsi. I get 18-20fps with this script. (it's a 6 core CPU and I've tried everything from 4 to 12 threads)

DGDecode_mpeg2source("C:\ADVENTURES_OF_SUPERMAN_DISC_3\VIDEO_TS\VTS_01_1.d2v", info=3)
TemporalDegrain(degrain=1,ov=2,hq=1)
RemoveDirtMC(25)
Greyscale()
Prefetch(8)

About 1 in 4 encodes will actually complete without hanging, which makes it essentially useless for me. Is this just the state of AVS+, or maybe a compatibility issue with plugins? Any ideas?
It might well be outdated plugins causing this.

First I suggest you post your list of plugins with version numbers. To do this easily, use AVSMeter (link in my sig) with the switches "-avsinfo -log". Save the log and post it.

Secondly, run the script with AVSMeter, check memory usage depending on the number of threads in your "Prefetch()" statement. Report back if it crashes.

[Bexley]

Here's the relevant output. I snipped the stuff after plugins. I installed new plugins from here (http://avisynth.nl/index.php/AviSynth%2B_x64_plugins) since it was a clean install and put them in the plugins+ directory. The only ones I reused from my old install were DGDecode, hqdn3d, RemoveDirt, and AVSRecursion.

Log file created with: AVSMeter 2.7.1 (x86)

[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 16299)
CPU brand string: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
CPU features: MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, FMA3, MOVBE, POPCNT, AES, F16C


[Avisynth info]
VersionString: AviSynth+ 0.1 (r2580, MT, i386)
VersionNumber: 2.60
File / Product version: 0.1.0.0 / 0.1.0.0
Interface Version: 6
Multi-threading support: Yes
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2017-12-28, 02:51:54 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins+


[CPP 2.5 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\DGDecode.dll [1.5.8.0]
C:\Program Files (x86)\AviSynth+\plugins\hqdn3d.dll [2005-01-25]
C:\Program Files (x86)\AviSynth+\plugins\RemoveDirt.dll [2005-05-07]

[CPP 2.6 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins+\ConvertStacked.dll [2017-12-28]
C:\Program Files (x86)\AviSynth+\plugins+\DePan.dll [2.13.1.3]
C:\Program Files (x86)\AviSynth+\plugins+\DePanEstimate.dll [2.10.0.2]
C:\Program Files (x86)\AviSynth+\plugins+\DirectShowSource.dll [2017-12-28]
C:\Program Files (x86)\AviSynth+\plugins+\fft3dfilter.dll [2.4.0.0]
C:\Program Files (x86)\AviSynth+\plugins+\ImageSeq.dll [2017-12-28]
C:\Program Files (x86)\AviSynth+\plugins+\masktools2.dll [2.2.12.0]
C:\Program Files (x86)\AviSynth+\plugins+\mvtools2.dll [2.7.24.0]
C:\Program Files (x86)\AviSynth+\plugins+\RgTools.dll [0.96.0.0]
C:\Program Files (x86)\AviSynth+\plugins+\Shibatch.dll [2017-12-28]
C:\Program Files (x86)\AviSynth+\plugins+\TimeStretch.dll [2017-12-28]
C:\Program Files (x86)\AviSynth+\plugins+\VDubFilter.dll [2017-12-28]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.avsi [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins+\mtmodes.avsi [2018-01-22]
C:\Program Files (x86)\AviSynth+\plugins+\TemporalDegrain.avsi [2016-11-22]
C:\Program Files (x86)\AviSynth+\plugins\RemoveDirtMC.avsi [2010-06-29]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\AvsRecursion.dll [2005-09-13]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2016-07-05]

[Groucho2004]

Looks all good to me. Now try my second suggestion.

[StainlessS]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\AvsRecursion.dll [2005-09-13]

system32.

EDIT: Yep, I mean SysWOW32.

[Bexley]

Seems to be completing OK. I made 2 passes with 8 and 12 prefetch threads, and it completed both times at right around 26fps with 27% CPU and 1139|1282 MiB memory usage. That seems about right based on past experience with 2.6. So I'm wondering now if AVS+ might be tickling something in x264 or MeGui.

[Bexley]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\AvsRecursion.dll [2005-09-13]

system32.

Ah yes, of course. Stupid mistake. But surely SYSWOW64 since it's a 32-bit file, right?

[Taurus]

Ah yes, of course. Stupid mistake. But surely SYSWOW64 since it's a 32-bit file, right?
Aah,
everybody is running in this trap sometimes:devil::)

[Bexley]

I spoke too soon. It hung on the 3rd run with 8 threads.

[Groucho2004]

I spoke too soon. It hung on the 3rd run with 8 threads.I assume no error message? Try to isolate the plugin that causes the problem by removing the filters one by one. Try without multi-threading.

[Bexley]

I've made about a dozen test runs of different scripts with AVSmeter, and although it doesn't really mean anything definitively, I have had 2 hangs when I include RemoveDirtMC(). No hangs with just TemporalDegrain(). So......

I know that RemoveDirt has a checkered past, so it doesn't surprise me that it might be the problem. So at this point I think I probably have 3 options. 1) go back to 2.6MT, 2) find a different version of removedirt.dll that plays nice, or 3) find a comparable function to RDMC() that gives a similar result. I'm open to suggestions.

And actually, everything I use except RDMC() seems to work great in 64-bit, so if I could find a similar function with 64-bit plugins it might allow me to just go 64-bit and be done.

[Groucho2004]

And actually, everything I use except RDMC() seems to work great in 64-bit, so if I could find a similar function with 64-bit plugins it might allow me to just go 64-bit and be done.
Post a short sample clip that shows the grain/dirt you want to remove. Someone may suggest an alternative to RemoveDirt.

I'm not sure if you have seen this already, here (http://avisynth.nl/index.php/External_filters#Film_Damage_correction) is a list of plugins/scripts with a similar purpose.

[Bexley]

Sample here (http://s000.tinyupload.com/index.php?file_id=94615222572802317661) showing before and after using the script above.

[Groucho2004]

Sample here (http://s000.tinyupload.com/index.php?file_id=94615222572802317661) showing before and after using the script above.Can you also post the unprocessed/uncompressed source sample?

[Bexley]

Can you also post the unprocessed/uncompressed source sample?

Yes, if I can find a free VOB splitter that isn't full of adware. That's not a standard tool in my toolbox.

[Groucho2004]

Yes, if I can find a free VOB splitter that isn't full of adware. That's not a standard tool in my toolbox.I use DGIndex (http://avisynth.nl/index.php/DGDecode) for that. The manual (http://rationalqm.us/dgmpgdec/DGIndexManual.html#Selection) explains how to select a part of the timeline. Then you can export it as an elementary stream (File -> Save project and demux video).

[Bexley]

I use DGIndex (http://avisynth.nl/index.php/DGDecode) for that. The manual (http://rationalqm.us/dgmpgdec/DGIndexManual.html#Selection) explains how to select a part of the timeline. Then you can export it as an elementary stream (File -> Save project and demux video).

Thanks, did not realize it did that. Untouched DVD sample here (http://s000.tinyupload.com/index.php?file_id=23596135505523523169).

[Groucho2004]

Thanks, did not realize it did that. Untouched DVD sample here (http://s000.tinyupload.com/index.php?file_id=23596135505523523169).
DeSpot() might be worth a try.