Avisynth+

[Myrsloik]

I HATE YOU ALL

The baked code related to PClip doesn't error out if AVS_Linkage is null. Instead it does nothing. This means that if a PClip (and probably most other objects too) is constructed with null linkage it'll consist of uninitialized memory and will most likely crash if the linkage is set later.

Just crash immediately if this happens, you'll save the world a lot of bugs that way.

[tormento]

no, no convert will done, input = output, but you can do it by your self, there are no need for dither dll without lsb in first place

A bit cryptic to me. :(

[real.finder]

A bit cryptic to me. :(

convertbits(16)
SMDegrain()
convertbits(10,dither=1,dither_bits=8)

[LigH]

Suddenly I am reminded of working in CoolEdit / Audition with Noise Shaping. Just visually now. :D

[real.finder]

I just find this by chance https://github.com/DJATOM/TCPDeliver/releases I put the link since there are no TCPDeliver in avs+

[Motenai Yoda]

A bit cryptic to me. :(

Sta senza pensieri. (cit)

[VS_Fan]

Suddenly I am reminded of working in CoolEdit / Audition with Noise Shaping. Just visually now. :DYou could try fft3dfilter with "noise pattern" (shape). From the description (http://avisynth.org.ru/fft3dfilter/fft3dfilter.html)of the filter:

Using noise pattern

Since v1.5 it is possible to get noise pattern (shape) by spectrum analysis of some empty block (without any objects), and then to reduce the noise with the same pattern in whole frame and in whole clip. It may be useful for removal of film (especially amateur 8 mm) grain and analog TV capture interference.


pframe - noise pattern frame number (default=false)
px - noise pattern block horizontal X position (default=0)
py - noise pattern block vertical Y position (default=0)
if px=px=0, then the pattern block is defined automatically with minimal power spectral density.
pshow - show noise pattern block and its properties (default=false)
pcutoff - noise pattern cutoff frequency (relative to max) (default=0.1)
pfactor - noise pattern denoise strength (0 to 1.0, default=0, this method disabled)


In order to use noise pattern method in place of ordinary (blind) method you must:
Firstly switch show mode pshow=true, and set some non-zero value pfactor=1.0.
Then select frame number and some block position, thus the block shown must not contain any objects beside typical noise pattern.
The switch off show mode pshow=false,
and set noise reduction strength, recomended value pfactor= 0.5 to 1.0.
The best windows type for this method is wintype=2.
The sigma and beta parameters are not used in this denoising method, but you can use this method in show mode pshow=true to estimate sigma value for ordinary denoising method (it is not strictly the same, but similar value).

[LigH]

Well, I just referred to working in high bit depth and finalizing with dithering.

[videoFred]

Hi Fred, welcome to Avisynth+! I hope we'll solve your problems.

That's very kind, thank you in advance :)


Can you check that you don't have another Prefetch somewhere else (for example in an imported avsi)?

No other Prefetch in the script itself and also not in imported avsi files, but I have found the problem. My script is using a function to select the different results with a AvsPmod slider. And here is the problem: with MT enabled I get the "prefetch" error message when selecting another result. So I have to disable MT, then select another result, then enable MT again.

Please try this script and you will get the error message too.

result = (selectresult([<"select result", 0, 3, 1>])) #function selectresult() see end of script

SetFilterMTMode("DEFAULT_MT_MODE", 2)

source= AviSource("Z:\VDP\VdP_Sp2.avi")

A = source
B = source.tweak(sat=2.0)
C = source.converttoRGB().GamMac()
D = source.sharpen(1.0)

eval(result)

[Multi Tasking=0]
Prefetch(4)
[/Multi Tasking]


function selectresult(int index)
{
return Select(index,"A","B","C","D")
}


For some reason prefetch must be called twice when switching results. But only with the r452 version of AvsPmod. It works fine with AvsPmod v2.5.1

Fred.

[pinterf]

Fred, I cannot help you with that at the moment.

Is there anybody out there who can tell that the compilation of the existing AvsPmod project is easy and straightforward?

Now I have some work with masktools, then it seems that there are still a couple of things to investigate.
- Scriptclip mt behaviour
- vdubfilter (deshaker issue at exit)
- PClip thing, that Myrsloik has kindly noticed and possibly presented him some happy hours and turned his hair grey.
- avspmod
...zimg

[MysteryX]

- vdubfilter (deshaker issue at exit)

Isn't that the issue I had fixed?

[olex99]

I've just moved to Avisynth+ from Avisynth 2.6 MT and i'm seeing some weird performance issues.

Using the following script in AviSynth 2.6 MT (x86), I get an average of 4.115fps

SetMemoryMax(2000)
SetMTMode(5, 6)

DGSource("D:\Videos\Video.dgi")

SetMTMode(2)
QTGMC(EdiThreads=1, DftThreads=1).SelectEven()
Trim(0,999)

Using a the following script in AviSynth+ r2455 (x64), I get an average of 2.505fps

SetMemoryMax(2000)

DGSource("D:\Videos\Video.dgi")
QTGMC(EdiThreads=1, DftThreads=1).SelectEven()
Trim(0,999)

Prefetch(6)

I've tried the 32bit version of avisynth+ and see similar performance to the 64bit version.

I've downloaded a MTModes.avsi file, aswell as trying to set the Default MT Mode to 2 and nothing seemed to make a difference.

Any ideas of why I might be seeing such a big difference in speed? I was hoping to move to Avisynth+ 64 bit to utilise more memory to increase performance, setting max memory to 3000 in Avisynth 2.6 MT gets me another 1fps but it crashes as soon as I try encode cause it runs out of memory.

The following plugins are installed (for both x64 and x86):
DGDecodeNV 0.0.0.2052
masktools2 - 2.2.4.0
mvtools2 - 2.7.15.22
nnedi3 - 0.9.4.37
rgtools - 0.95.0.0

Thanks

[pinterf]

Isn't that the issue I had fixed?
Yes, you have partially fixed an issue (https://github.com/pinterf/AviSynthPlus/commit/e5e8bbdca48d9fbd561ae6c6a0f1e90760460a9c), but now it was reported again, this time with a multiple call scenario

Works:
source="test.mp4"
LoadVirtualDubPlugin ("c:\Virtualdub\plugins32\deshaker.vdf", "deshaker", preroll=0)
FFVideoSource(source)
clip=ConvertToRGB32()
clip.deshaker("
12|2|30|4|1.09402|1|1|0|640|480|
1|2|1|400|400|400|1500|4|1|4|2|
5|40|300|4|C:\deshaker.log|0|0|0|0|0|
0|0|0|0|0|0|0|0|1|15|
15|5|15|1|1|30|30|0|0|0|
0|1|0|1|10|1|15|1000|1|88")


Freeze on exit:

source="test.mp4"
LoadVirtualDubPlugin ("c:\Virtualdub\plugins32\deshaker.vdf", "deshaker", preroll=0)
FFVideoSource(source)
clip=ConvertToRGB32()
clip.deshaker("
19|1|30|4|1|0|1|0|1920|1080|
1|2|1000|1000|1000|1000|4|1|0|2|
8|30|300|4|C:\Deshaker.log|0|0|0|0|0|
0|0|0|0|0|0|0|0|1|15|
15|5|15|0|0|30|30|0|0|1|
0|1|0|0|10|1000|1|104|1|1|
20|5000|100|20|1|0|ff00ff")

clip.deshaker("
19|2|30|4|1|0|1|0|1920|1080|
1|2|1000|1000|1000|1000|4|1|0|2|
8|30|300|4|C:\Deshaker.log|0|0|0|0|0|
0|0|0|0|0|0|0|0|1|15|
15|5|15|0|0|30|30|0|0|1|
0|1|0|0|10|1000|1|104|1|1|
20|5000|100|20|1|0|ff00ff")

The difference is that we have two deshaker calls.

[LigH]

^ which appears nonsensical to me, especially if you take the implicit assignment to last into account:

...
clip = ConvertToRGB32()
last = clip.deshaker({ParamSet1}) # ignored due to the following call superseding the output
last = clip.deshaker({ParamSet2}) # this is the call producing the script output

If you want to create a sequence of two calls, which possibly depend on each other, you will have to ensure that both will have an impact on the output clip, and may it just be a merge of 0% and 100% weight. Furthermore, understand that the sequence will run per frame: If you need a first pass to produce a log file, and a second pass to read and process it, you will need two different scripts anyway, or the second call will not find a finished log file because the first call did not yet complete it.

[pinterf]

^ which appears nonsensical to me, especially if you take the implicit assignment to last into account:

Report is from here (https://forum.doom9.org/showthread.php?p=1801511#post1801511)
Anyway, it shouldn't freeze on exit.

[LigH]

I'll post my thoughts there if not yet answered by someone else; for me it looks like the author of this report tried to run a 2-pass sequence in one script, so the second-pass call will have either missed a non-existing log file, or tried to read from an open file being written to (and depending on file sharing modes, this might cause a lock?). So I would not bet on the AviSynth core to be blamed.

[StainlessS]

Small point though, the first calls constructor will be called, even if that filters result is ignored.
EDIT: Is deshaker re-entrant. Perhaps adding a "Last=0" between calls would make freeze disappear, ie call destructor on Last.

EDIT: Post here seems to suggest that log does indeed need to be closed between calls (as per LigH):- https://forum.doom9.org/showthread.php?p=1782998#post1782998
Also of course need complete scan between calls.

[pinterf]

I've just moved to Avisynth+ from Avisynth 2.6 MT and i'm seeing some weird performance issues.

Using the following script in AviSynth 2.6 MT (x86), I get an average of 4.115fps
[...]
Using a the following script in AviSynth+ r2455 (x64), I get an average of 2.505fps
[...]
Thanks
For Prefetch(6) the SetMemoryMax(2000) kills performance for this script.
Avisynth+ r2455:
x64, SetMemoryMax(2000): 6.1 fps
x64, SetMemoryMax(3000): 8.8 fps (AVSMeter: 2500MB virtual)
x86, SetMemoryMax(2000): 7.7 fps
x86, SetMemoryMax(3000): 8.2 fps (AVSMeter: 2600MB virtual)

AVS2.6 MT 2.6.0.5
x86, SetMemoryMax(2000): 8.7 fps (AVSMeter: 2540MB virtual)
x86, SetMemoryMax(3000): 8.6 fps (AVSMeter: 3540MB virtual)

I don't have DGSource, used a test clip with lsmashvideosource.
Run from AVSMeter, no extra memory was needed for the encoder.

Using this script line:
SetLogParams("log.txt", LOG_DEBUG)
a warning appears in the log file for the SetMemoryMax(2000) case:
WARNING: Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax().

[olex99]

Thanks Pinterf.

You are right, i've managed to get similar performance between Avisynth+ 64bit and Avisynth 2.6 MT by setting the Max Memory to 3000.

I did a lot of testing a few months back on the best Max Memory value for my setup and I settled on 2000 as it gave me the best speed and reliability, going to 3000 in x86 obviously gave me faster speed but it would crash when encoding cause the process would run out of memory.

I'm surprised to see such a big performance difference between Avisynth+ and Avisynth 2.6 MT, the same script and plugins are about 40% faster in Avisynth 2.6 MT, I guess the internals of both programs are a fair bit different though when it comes to multithreading.

The bonus of Avisynth+ is the 64bit mode though which allows me to pump more memory into it but it looks like for my setup, 6 threads and 3000 max memory give me the best performance, anymore threads and all it does is increase the amount of memory required without giving me any performance increase.

Is there anything you can think of that might increase my performance in Avisynth+ to get it closer to Avisynth MT?

I am looking at getting some more ram soon as i'm currently stuck running single channel on a x58 xeon which can run triple channel and I've found memory speed gives a fairly substantial jump in performance so hopefully when I get that I should see a difference.

Thanks for your help.

[vdcrim]

AvsPmod throws this error message:
Only a single prefetcher is allowed per script
But the only prefetch in the script is this one.

It's a bug in AvsPmod, I just posted a build with a fix here (https://forum.doom9.org/showpost.php?p=1801766&postcount=1202).

[pinterf]

I'm surprised to see such a big performance difference between Avisynth+ and Avisynth 2.6 MT, the same script and plugins are about 40% faster in Avisynth 2.6 MT, I guess the internals of both programs are a fair bit different though when it comes to multithreading.

40% difference between avs+ 32 and avs+ x64 (and classic x86 Avisynth MT 2.6.0.5) is too huge and it cannot be reasoned by internal mt differences.

[olex99]

40% difference between avs+ 32 and avs+ x64 (and classic x86 Avisynth MT 2.6.0.5) is too huge and it cannot be reasoned by internal mt differences.

Would you expect the 64bit version to be faster than the 32bit version? I know x264 is meant to be faster as 64bit and I assumed that Avisynth would be the same but from what I've seen in my tests the 32bit version is faster.

I don't have the results of my tests on me as the computer is at work so ill have to get them on Monday but I ended up getting the speed to within 5% of each other. Avisynth 2.6 MT x86 with SetMaxMemory at 2000 is about 5-10% faster than Avisynth+ 64 bit with SetMaxMemory at 3000.

Today I basically uninstalled Avisynth+ and normal Avisynth, deleted everything and started from scratch. Installed Avisynth+ 2294 and then updated to 2455, removed all plugin other than the ones I use for QTGMC. I downloaded the latest QTGMC 3.355s as well as the latest SMDegrain.

Then I ran 1000 frames of a 1080i (25fps) video through AVSMeter with a whole bunch of different SetMaxMemory and Prefetch, trying to match or beat the speed I got with Avisynth 2.6MT.

One thing I haven't tried is Avisynth+ x86 version to see what sort of speed I get from that but I was hoping to use the x64 version so I could use more memory as I know 2000 is limiting but cant go anymore than that in x86 without it crashing.

I've set the logging in Avisynth+ to Debug and it is empty.

[pinterf]

Would you expect the 64bit version to be faster than the 32bit version? I know x264 is meant to be faster as 64bit and I assumed that Avisynth would be the same but from what I've seen in my tests the 32bit version is faster.

mvtools2, RgTools benefit from running on x64.
In QTGMC most of the time is spent in plugins, not in avisynth, unless there are other factors like mt sceduling and memory issues.
Btw, on what exact processor type have you made the speed tests?

[olex99]

mvtools2, RgTools benefit from running on x64.
In QTGMC most of the time is spent in plugins, not in avisynth, unless there are other factors like mt sceduling and memory issues.
Btw, on what exact processor type have you made the speed tests?

Is there a way I can work out which plugin/s might be the issue?

It's weird cause i'm literally using the exact same plugins and scripts between the different avisynths, I downloaded all the latest ones this morning. I installed Avisynth+ 2294 exe and then copied the Avisynth.dll and DevIl.dll from Avisynth 2.6 MT over the top of the x86 version and the same dlls from Avisynth r2455 over the top of the x64 version..

I'm running an Intel Xeon X5675 6 core clocked at 4ghz with 8gig of ddr3 ram clocked at 1820mhz with Windows 10 Pro x64. As I said in a previous post, the ram is only running single channel at the moment but I'm looking at buying another 2 sticks for triple channel in the next week or so, I found the faster I have the memory clocked the faster the encoding runs.

The scripts are literally just doing a load of the source using DGDecodeNV and then QTGMC with default settings other than setting the threads to 1.. I've commented the QTGMC call out to see if there was an issue with DGDecodeNV, however both versions of avisynth give me around 340fps with just DGDecode in there so its got to be something within QTGMC.

[olex99]

I wonder if it could be something to do with the runtime version I'm using? I can't remember the versions but I installed the x86 ones a while ago and I did have to install a x64 version for one of the plugins so maybe there is a difference there.

On Monday when I get back to the computer I'll do some testing between avisynth 2.6 mt and avisynth+ x86 and see what the performance difference is. Least that way I can narrow down whether it's a difference between avisynth and avisynth+ or if there is something wrong with my x64 setup.

Should I expect them to be pretty similar with the exact same plugins and script, the only difference in script would be the different way we specify MT.

[DJATOM]

I'm running an Intel Xeon X5675 6 core clocked at 4ghz with 8gig of ddr3 ram clocked at 1820mhz with Windows 10 Pro x64

My friend has encoding server with the same CPU, but there are actually 2 CPUs and he didn't OC them. Also we using windows server 2008 as host OS.
All I can say that's a good low-cost CPU for encoding, especially if you can afford a pair of them. It's above 2x speed boost against my i5-4670k clocked at 4.3 GHz.

pinterf
If you need any test on that CPU, ask me in PM or so.

[pinterf]

Thanks, I was just wondering whether the processor has AVX or better capabilities (but not, it has only SSE4.2), because Avs+ can report AVX or better CPU flags to plugins, and that may result in a different code path. And if that code path would contain bug that can explain the different.

Another question for olex99, please set SetMemoryMax to 5000, and run the x64 avsmeter64 process to see how much memory is needed actually (it tops at a maximum, I don't expect to reach 5000). Maybe even 3000 is not enough (though you have said that no cache warning was seen in your logs). What are the frame dimensions (and format YV12?) of your clip?

[pinterf]

For those who are intested in what happens in the background, this is what Visual Studio performance profiler shows, Avs+ x86.

Script
SetMemoryMax(3000)
SetFilterMTMode("DEFAULT_MT_MODE",2)
lsmashvideosource("test107frame.mp4", format="YUV420P8").Loop(10)
Crop(0, 140, 0, -140)
QTGMC(EdiThreads=1, fftThreads=1).SelectEven()
Trim(0,999)
Prefetch(6)


Function Name Inclusive Samples % Exclusive Samples % Module Name
[nnedi3.dll]
6,76 6,76 nnedi3.dll
Filtering::MaskTools::Filters::Lut::Dual::lut_c
5,82 5,82 masktools2.dll
[LSMASHSource.dll]
5,83 5,79 LSMASHSource.dll
resizer_h_ssse3_generic
5,65 5,65 avisynth.dll
Degrain1to6_sse2<16,16,0,1>
5,09 5,09 mvtools2.dll
_Overlaps16x16_sse2
4,60 4,60 mvtools2.dll
memcpy
4,32 4,32 vcruntime140.dll
_VerticalWiener_iSSE
4,12 4,12 mvtools2.dll
Short2Bytes
4,00 4,00 mvtools2.dll
PlaneOfBlocks::PseudoEPZSearch<unsigned char>
3,44 3,44 mvtools2.dll
_HorizontalWiener_iSSE
3,28 3,28 mvtools2.dll
_x264_pixel_sad_16x16_sse2
2,81 2,81 mvtools2.dll
_Overlaps8x8_sse2
2,50 2,50 mvtools2.dll
_x264_pixel_sad_8x8_mmx2
2,32 2,32 mvtools2.dll
resize_v_ssse3_planar<&simd_load_streaming>
2,31 2,31 avisynth.dll
Degrain1to6_sse2<8,8,0,1>
2,10 2,10 mvtools2.dll
MVDegrainX::process_chroma
4,19 2,10 mvtools2.dll
PlaneOfBlocks::search_mv_slice<unsigned char>
1,95 1,94 mvtools2.dll
PlaneOfBlocks::ExpandingSearch<unsigned char>
1,89 1,89 mvtools2.dll
PlaneOfBlocks::FetchPredictors<unsigned char>
1,76 1,76 mvtools2.dll
_Copy16x16_sse2
1,71 1,71 mvtools2.dll
_Thread32Next@8
1,59 1,59 kernel32.dll
process_plane_sse<unsigned char,&rg_mode20_sse<0,1>,&rg_mode20_sse<1,1> >
1,43 1,43 RgTools.dll
MVDegrainX::GetFrame
37,15 1,28 mvtools2.dll
Filtering::MaskTools::Filters::Morphologic::xxpand_sse2_vertical
<&expand_operator_sse2,&Filtering::MaskTools::Filters::Morphologic::limit_up_sse2,1>
1,26 1,26 masktools2.dll
weighted_merge_planar_sse2
1,26 1,26 avisynth.dll
Filtering::MaskTools::Filters::Morphologic::xxpand_sse2_vertical
<&inpand_operator_sse2,&Filtering::MaskTools::Filters::Morphologic::limit_down_sse2,1>
1,22 1,22 masktools2.dll
PlaneOfBlocks::InterpolatePrediction<unsigned char>
1,20 1,20 mvtools2.dll
_Copy8x8_sse2
1,19 1,19 mvtools2.dll
calculate_sad_sse2<0>
0,76 0,76 avisynth.dll
PlaneOfBlocks::Hex2Search<unsigned char>
0,69 0,69 mvtools2.dll
accumulate_line_sse2<1,1>
0,66 0,66 avisynth.dll
MVCompensate::compensate_slice_overlap
0,65 0,65 mvtools2.dll
logic_t_sse2<1,&max_t_sse2<&nop_sse2,&nop_sse2>,&max_t<&nop,&nop> >
0,62 0,62 masktools2.dll
process_plane_sse<unsigned char,&rg_mode11_sse<0,1>,&rg_mode11_sse<1,1> >
0,62 0,62 RgTools.dll
Filtering::MaskTools::Filters::Lut::Single::lut_c
0,60 0,60 masktools2.dll
norm_weights<1>
0,57 0,57 mvtools2.dll
logic_t_sse2<1,&min_t_sse2<&nop_sse2,&nop_sse2>,&min_t<&nop,&nop> >
0,57 0,57 masktools2.dll
Filtering::MaskTools::Filters::Support::MakeDiff::makediff_sse2_t<1>
0,48 0,48 masktools2.dll
_RB2BilinearFilteredVerticalLine_SSE
0,41 0,41 mvtools2.dll

[videoFred]

It's a bug in AvsPmod, I just posted a build with a fix here (https://forum.doom9.org/showpost.php?p=1801766&postcount=1202).

Thank you!

Fred.

[MysteryX]

Pinterf, here's a bug in Avisynth+. Better if I leave this one for you so it's done the right way instead of hacking around the issue.
https://forum.doom9.org/showthread.php?t=174459

[real.finder]

hi Pinterf

ColorYUV not work with float yet

I usually use it with ColorYUV(autogain=true)

and levels="TV->PC"

[pinterf]

Yes. This is a lut using filter, which is not available in float. O.k. I will make it work realtime.
Now I'm still working on masktools, finally decided to implement dup and swap in expressions as you have requested.

[olex99]

Thanks, I was just wondering whether the processor has AVX or better capabilities (but not, it has only SSE4.2), because Avs+ can report AVX or better CPU flags to plugins, and that may result in a different code path. And if that code path would contain bug that can explain the different.

Another question for olex99, please set SetMemoryMax to 5000, and run the x64 avsmeter64 process to see how much memory is needed actually (it tops at a maximum, I don't expect to reach 5000). Maybe even 3000 is not enough (though you have said that no cache warning was seen in your logs). What are the frame dimensions (and format YV12?) of your clip?


Hi Pinterf, I've finally got back to the computer and managed to run some more tests. I upped the Max Memory to 6000 as at 3000 it was still showing the cache warning (although I'm sure I checked that), and they are definitely much closer to AVS 2.6 MT now. The source clup is 1080i/25 recording, it is reported as YUV in MediaInfo, however Avisynth reports it as YV12 using both DGSource and LWLibAVVideoSource.

Here are my results:


AviSynth 2.6 x86 (6000 Memory/6 Threads)

AviSynth 2.60, build:Feb 20 2015 [03:16:45] (2.6.0.5)

Number of frames: 1001
Length (hh:mm:ss.ms): 00:00:40.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420
Active MT Mode: 2
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a

Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.746 | 54568 | 4.025
Memory usage (phys | virt): 3669 | 3800 MiB
Thread count: 33
CPU usage (average): 42%

Time (elapsed): 00:04:08.668




AviSynth+ x64 (6000 Memory/6 Threads)

AviSynth+ 0.1 (r2455, MT, x86_64) (0.1.0.0)

Number of frames: 1001
Length (hh:mm:ss.ms): 00:00:40.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a

Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.323 | 33870 | 3.846
Memory usage (phys | virt): 3511 | 3589 MiB
Thread count: 45
CPU usage (average): 43%

Time (elapsed): 00:04:20.292



Avisynth+ x86 (6000 Memory/6 Threads)

AviSynth+ 0.1 (r2455, MT, i386) (0.1.0.0)

Number of frames: 1001
Length (hh:mm:ss.ms): 00:00:40.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a

Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.371 | 34464 | 4.083
Memory usage (phys | virt): 3536 | 3635 MiB
Thread count: 45
CPU usage (average): 43%

Time (elapsed): 00:04:05.167



It seems AviSynth+ is much more susceptible to low memory than AviSynth is so looks like I've just got to give it more to keep it happy, interestingly too AviSynth+ uses 12 more threads than AviSynth.

Is there anything I can do to get the x64 version running faster, even with a SetMaxMemory of 6000, x64 is slower than AVS 2.6MT with a SetMaxMemory of 2000. On the plus side, it is much more stable as I was regularly getting crashes with AVS 2.6MT, most likely due to running out of memory.

Here is my log for the AVS 2.6MT at SetMaxMemory 2000

AviSynth 2.60, build:Feb 20 2015 [03:16:45] (2.6.0.5)

Number of frames: 1001
Length (hh:mm:ss.ms): 00:00:40.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420
Active MT Mode: 2
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a

Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.717 | 60444 | 4.093
Memory usage (phys | virt): 2620 | 2715 MiB
Thread count: 33
CPU usage (average): 43%

Time (elapsed): 00:04:04.555

Thanks for your help

[tuanden0]

I'm using this chain here (https://forum.doom9.org/showthread.php?p=1091224#post1091224) to upscale my 576p video to 720p and change some filter.
Here's my script:
SetMemoryMax(8000)
SetFilterMtMode("MP_Pipeline", MT_SPECIAL_MT)
MP_Pipeline("""
### platform: win64
LWLibavVideoSource("E:\Download\test.mkv")
AssumeFPS(24000, 1001)
### ###
### platform: win32
Toon()
### ###
### platform: win64
Deblock(quant=33)
FluxSmoothST()
Spline64ResizeMT(1536,864)
### ###
### platform: win32
aWarpSharp(depth=12,blurlevel=4,thresh=0.3,cm=1)
LSFmod(edgemode=1,strength=200)
### ###
### platform: win64
FFT3dFilter(sigma=3.2, bt=1, ncpu=4)
### ###
### platform: win32
Dehalo_alpha()
### ###
### platform: win64
Spline64ResizeMT(1280,720)
### ###
""")
Then, Can someone help me to put Prefetch(4) to this script? :thanks:
I tried to put it into my script but it crashed.

[ChaosKing]

mp_pipeline has it's own prefetch function, see example here: https://forum.doom9.org/showthread.php?t=163281

btw aWarpSharp, dehalo_alpha and probably some other filters are x64 compatible...

[djonline]

I also have problem with deshaker on second pass, both x32 and x64. AvisynthPlus-r2455-MT.
1. If I simple call deshaker plugin, there is only one repeated frame in final video.
00114.MTS-pass2-64.avs

SetMemoryMax(3000)
vid="00114.MTS"
o=DirectShowSource(vid).ConvertToRgb32(matrix="Rec709")
o+o.Trim(0,29)
LoadVirtualDubPlugin ("c:\Program Files\Vdub\vdub64\plugins\Deshaker_64.vdf", "deshaker",0)
deshaker ("18|2|30|4|1|0|1|0|640|480|1|2|1000|1000|2000|2000|4|0|0|2|8|30|300|3|00114.MTS.0.1000.1000.2000.2000.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|70|70|10|30|1|1|30|30|0|0|0|0|1|0|1|10|1000|1|88|1|0|20|5000|100|20|1")

If I call SetMemoryMax(6000), there is only blank screen.

If I call this avs from another avs to trim first 30 frames, vdub is crashed.
DirectShowSource("00114.MTS-pass2-64.avs").Trim(30,0)

UPDATE: Sorry, I forgot that MTS in avisynth not work with LAV splitter, so I install Haali media splitter and now all work ok, both x32 and x64 Deshaker in Avisynth+. Benchamark at https://forum.doom9.org/showpost.php?p=1802602&postcount=3192

[StainlessS]

@ djonline

I have a very OLD version of Deshaker, and it is version 19 [first arg in your list is 18].

Why do you think that you need SetMemoryMax(3000) with that script, [god forbid SetMemoryMax(6000)] ?
Script as given would not work (something like "Script does not return a clip" would be issued).
Where is the rest of the script ? (aint nobody gonna be able to help unless they see the script).

EDIT: https://forum.doom9.org/newreply.php?do=newreply&p=1645218

And the SetMemoryMax(2048) is a ridiculous value. As I keep telling people setting the size of the Avisynth frame cache bigger than what the current script needs is a waste.

EDIT: Also, on XP32 with 4GB, SetMemoryMax(2000) can sometime run out of memory (seldom), [or SetMemoryMax(1000), or SetMemoryMax(500)]
strangely, on P4 XP32 with 1GB, I've never once had an OMEM, ever (with defaulted memory max, EDIT: Unless down to eg plugin bug).

EDIT: At the very least you need a return o at end of script, also, good idea to wrap your script in CODE tags, click Advanced, select the script, click on hash (#).

EDIT: Just noticed this :)

If I call this avs from another avs to trim first 30 frames, vdub is crashed.
DirectShowSource("00114.MTS-pass2-64.avs").Trim(30,0)

Dont think you would have too much luck loading an Avisynth script via DirectShowSource, Try Import() instead.
Also, you add 30 frames to END of original clip with o+trim(0,29)
and then DirectShowSource("00114.MTS-pass2-64.avs").Trim(30,0) which trims off the FIRST 30 frames, is that intentional ?

[djonline]

There is last Deshaker 3.1. I think first argument, 18 or 19, is not used.
Of course I try with and without SetMemoryMax.
This is full script, no any other lines. May be you don't have 00114.MTS.0.1000.1000.2000.2000.log on this second pass when you try it.
I have windows 10.
vdub.vcf

VirtualDub.Open("00114.MTS-loader.avs", 0, 0);
VirtualDub.audio.SetSource("00114.MTS.ac3");
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetClipMode(1, 1);
VirtualDub.audio.SetConversion(0, 0, 0, 0, 0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.audio.filters.Clear();
VirtualDub.audio.SetInterleave(1, 500, 1, 0, 0);
VirtualDub.video.SetDepth(24, 24);
VirtualDub.video.SetOutputFormat(0);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate2(0,0,1);
VirtualDub.video.SetIVTC(0, 0, -1, 0);
VirtualDub.video.SetCompression(0x7967616d,0,10000,0);
VirtualDub.video.SetCompData(68,"CAAAAAgAAAABAAAABAAAAAIAAAD/////AAAAAAIAAABpAAAAAAAAABkAAAAJAAAAAQAAAAEAAAAAAAAAAAAAAAcAAAAAAAAAAAAAAAAAAAABAAAA");
VirtualDub.video.filters.Clear(); VirtualDub.SaveAVI("00114.MTS-deshaked.1.2000.2000.2000.2000.avi");
VirtualDub.Close();


00114.MTS-loader.avs

DirectShowSource("00114.MTS-pass2-64.avs").Trim(30,0)

I use option "Use previous and future frames to fill in borders", so I must trim first 30 frames and add last 30 frames. But there is no way to do this in ony avs script, Deshaker always add first 30 empty frames, even I if add Trim(30,0) after Deshaker call, so I must use another script named 'loader'.
Vdub crush info in attach.

[LigH]

So you use VirtualDub to load an AviSynth script that loads another AviSynth script via DirectShowSource... :eek: :confused:

[StainlessS]

There is last Deshaker 3.1. I think first argument, 18 or 19, is not used.

The version number tells Deshaker what version script it is, so it will complain if wrong version, it should be for the version that you use,
which apparently is version 18 (not latest for sure).

Suggest that you change

DirectShowSource("00114.MTS-pass2-64.avs").Trim(30,0)

to

Import("00114.MTS-pass2-64.avs").Trim(30,0)

We still dont know what is in that 00114.MTS-pass2-64.avs script, if it is a Deshaker pass 1 script, it aint gonna work,
because pass 1 create Deshaker log for 2nd pass of Deshaker, and is not available until pass 1 has completed.

Also, note, if you have

o=DirectShowSource(vid).ConvertToRgb32(matrix="Rec709")
o+o.Trim(0,29)

in your first pass, and then use add a Trim(30,0) to end first pass before doing 2nd pass, then you Deshaker log will be 30 frames in error
and deshaker will dehake almost everything wrongly (as its using deshaker data for the wrong frames).

EDIT: Sorry, you said that this is the script,

SetMemoryMax(3000)
vid="00114.MTS"
o=DirectShowSource(vid).ConvertToRgb32(matrix="Rec709")
o+o.Trim(0,29)
LoadVirtualDubPlugin ("c:\Program Files\Vdub\vdub32\plugins32\Deshaker.vdf", "deshaker",0) deshaker
("18|2|30|4|1|0|1|0|640|480|1|2|1000|1000|2000|2000|4|0|0|2|8|30|300|3|00114.MTS.0.1000.1000.2000.2000.log"
"|0|0|0|0|0|0|0|0|0|0|0|0|0|1|70|70|10|30|1|1|30|30|0|0|0|0|1|0|1|10|1000|1|88|1|0|20|5000|100|20|1")


NOTE, That is a PASS 2 Script.

EDIT: Can you just state in words what it is that you are trying to do.

EDIT: OK, re-read everything, think I understand. Just Load the script above into VirtualDub and forget altogether about the "00114.MTS-loader.avs" script (But reduce the massive SetMemoryMax thing)

[djonline]

Sorry, I forgot that MTS in avisynth not work with LAV splitter, so I install Haali media splitter and now all work ok, both x32 and x64 Deshaker in Avisynth+.
So now I can post my benchmarks of x32 and x64 Deshaker:
1 pass: x32 - 36s, x64 - 21s, speedup 71%
2 pass :x32 - 30s, x64 - 20s, speedup 50%

Source 1080/60p from Sony RX100 28mbit, 8s length, output to MagicYUV 4:2:0 lossless.

[real.finder]

I get error like this https://forum.doom9.org/showpost.php?p=1779374&postcount=2372

with TDecimate and MT, is this bug didn't fixed yet?

[DJATOM]

I confirm that.
As a temporary solution I do tfm/tdecimate filtering via x86 avs+ and w/o Prefetch, transmit output with TCPDeliver, process anything else with x64 avs+ (and Prefetch).

[real.finder]

I confirm that.
As a temporary solution I do tfm/tdecimate filtering via x86 avs+ and w/o Prefetch, transmit output with TCPDeliver, process anything else with x64 avs+ (and Prefetch).

I am do same thing but with mp_pipeline

[yup]

Hi all!
I am new in high bit conversion, advice how convert from avs+ 16 bit to Avisynth 16 bit hack (for use inside KNLMeansCL plugin) and back. I could use dither or last avs+ make this by self?
yup.

[DJATOM]

Do your 16bit native filtering
Converttostacked()
Do stacked filtering
Convertfromstacked()
Do another HBD native filtering

It's simple as that.

[yup]

DJATOM! :thanks: yup.

[yup]

Hi all!
Advice also easy way for import hi bit image to Avisynth+.
yup.

[LigH]

You mean, reading a (sequence of) 48 bit PNG/TIFF as source (like raw footage of Blender render movies)?

[yup]

LigH!
Yes! For starting point I want filtering film scanned grey scale 16 bit image using non local means filter.
yup.