Log in

View Full Version : AviSynth+ thread Vol.2


Pages : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 [20] 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75

FranceBB
9th May 2021, 10:25
Nope, it doesn't happen on my end.
The only time it does actually happens is if I'm encoding with an executable remotely located on the network and there's a network problem on a switch, but again that's a rare scenario.

Can you try to give your AVS Script a spin with AVSMeter and see if it happens there as well?

Morku
9th May 2021, 11:12
Yes, it also happen in AVSMeter. But instead of rising ETA, adjust FPS cur = 0 (like MeGUI, VDub2) all counters just stop. "Freeze". And pressing Esc at this step, won't cancel the process and go back to cmd start. The point (processed frames) is completely random, but for this example mostly somewhere at the beginning.
Tried now an utvideo version 20.6.1 from a year ago and shorten the QTGMC to "QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)", but still the same issue.

https://i.imgur.com/oQF7x3Z.png

It's odd...

manolito
9th May 2021, 11:36
How many AVS+ MT threads are you using? I don't see a Prefetch command in your script.

If you use more than 8 threads then I would try to lower the thread number in the prefetch command.

Another thing which worked wonders for me is to force linear access to some filters. I would add
RequestLinear(rlim=50, clim=50)
right after the source filter and also after the QTGMC call because dfttest needs fftw3 which is not thread safe.

FranceBB
9th May 2021, 11:41
Another thing which worked wonders for me is to force linear access to some filters. I would add
RequestLinear(rlim=50, clim=50)
right after the source filter

Yep it helped me long ago for a thing, so I would recommend it too.

@Morku... let us know, but this is indeed odd. I'm trying to pinpoint whether it's a decoding issue or if one of the filters is to blame.
Out of pure curiosity, can you make some more tests?

Test 1:

AVISource("a.avi", audio=true)

just that to see if it stops.
If it doesn't, then it's not the source, but it's one of the filters and indeed RequestLinear() as suggested by Manolito can help, so make another test with it.

Test2:
AVISource("a.avi", audio=true)
RequestLinear(rlim=50, clim=50)

#rest of the script

as suggested by Manolito.
It will slow things down a bit, but it might help.


Test 3:

FFMpegSource2("a.avi", atrack=-1)

what happens if you swap the indexer? (I know that UTVideo videos are not supposed to be indexed, but again, I'm trying to pinpoint what's causing the issue).

Feel free to make all these tests in AVSMeter so that you don't have the additional nag on your CPU to actually encode a file...

Morku
9th May 2021, 17:24
@manolito
How I can find out? I am not that knowledged that way.

Test 1: Finished successfully: https://i.imgur.com/ZOSMxfp.png
Test 2: Stopped at a very late point this time. Might be coincidence? https://i.imgur.com/yOqqYvi.png
global MeGUI_darx = 160
global MeGUI_dary = 117

AVISource("3.avi", audio=true).AssumeFPS(25,1)
RequestLinear(rlim=50, clim=50)

AssumeTFF()
top = Crop(0,0,-0,288)
bot = Crop(0,289,-0,-0).AddBorders(0,0,0,1)

StackVertical(top,bot)

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

ConvertToYV12()

__film = last
__t0 = __film.trim(134, 281048)
__t0
Test 3: Stopped at a very early point... https://i.imgur.com/FpjF9Dn.png
global MeGUI_darx = 160
global MeGUI_dary = 117

FFMpegSource2("3.avi", atrack=-1).AssumeFPS(25,1)
RequestLinear(rlim=50, clim=50)

AssumeTFF()
top = Crop(0,0,-0,288)
bot = Crop(0,289,-0,-0).AddBorders(0,0,0,1)

StackVertical(top,bot)

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

ConvertToYV12()

__film = last
__t0 = __film.trim(134, 281048)
__t0

StainlessS
9th May 2021, 17:31
You could start by removing lines from the script,eg
MeGUI darx/y dont really need to be in there, and just adds extra fluff to the mix.

How bout

FFMpegSource2("3.avi", atrack=-1).AssumeFPS(25,1)

AssumeTFF()

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

ConvertToYV12()



If works ok, then start adding stuff back until problem. [EDIT: add back trims last]
EDIT: What version AVS+, and what version QTGMC.

EDIT: Is the height of the original clip ODD ?
(why add 1 to bottom, and deinterlace on originally odd height ???)
EDIT: No, I'm wrong, on bot clip you crop top scanline of bot, then add line to bottom of bot, but still a bit weird.

EDIT: Avoid RequestLinear if you are trying to find the bug [rather than hide it].

Morku
9th May 2021, 19:12
Tried now the very basic:
AVISource("3.avi", audio=true).AssumeFPS(25,1)

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

and it stopped at 12min proceed video.

Avisynth is 3.7.0_20210111 and QTGMC v3.379, Zs_RF_Shared v1.152.

Well the sourcevideo is even and odd. I have to add bottom one, because cropping 289 won't work for chroma resolution I think. But this seems not to be the reason.

I wouldn't wonder if this is just another Windows 2004 issue. This release brought other bugs. Worst release ever.
Otherwise I would expect an error. I can even eject the HDD, because it's not in use anymore. All the Avisynth programs won't notice that the source is missing when the issue occur. Can't wait to see 21H2.
I can't remeber to had an issue like that with 1909 and prior und use the script for years (since Windows 8).

I might test older AVS releases and if I can find for QTGMC.

StainlessS
9th May 2021, 20:29
I've tried MEGUI [probably not an up to date version] on W10, and every now and again it just disappears with no error in log, think there was some AVS wrapper problem or something with current AVS+].

and it stopped at 12min proceed video.
MeGUI or AvsMeter, or what
Yes, maybe try alternate version of QTGMC..

I'm currently screwing around with W10, trying to install 1909 on Linx 7 (1GB RAM, Win8.1) tablet, no memory for RAM disk error.
Can update from within W8.1, but not clean install from USB ISO.

real.finder
9th May 2021, 20:33
Tried now the very basic:
AVISource("3.avi", audio=true).AssumeFPS(25,1)

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

try with ColorBars(width=720, height=576, pixel_type="yv16").AssumeFPS(25,1)

QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

Morku
9th May 2021, 22:21
I reverted to AviSynth 3.5.1_20200402 and QTGMC 3.364 and it was looking promissing for a long time... but "failed" (stopped) again almost near the goal :( https://i.imgur.com/00NuHgZ.png

@StainlessS
All my last tests were with AVSMeter and I whish there would be error logs in MeGUI. But the Status window just stay there, Time remaining rise up, Current frames stop at a random point. Like an endless encoding.

Tomorrow I will update AVS and QTGMC to latest and test real.finder advise, but I don't have high hopes. Thanks to all.

real.finder
9th May 2021, 22:54
in general the problems come from dlls not avsi, QTGMC is avsi so update it or downgrade it probably will not do big change in this case

LigH
10th May 2021, 08:03
@Morku:

Just to add confidence in the source: VirtualDub has a "Scan for errors" command in the "Video" menu, which will decode the source quickly without any processing, useful for AVIs in doubt.

But if you say it's a random position, it is probably rather some processing issue. Does it completely stop, or is the harddisk very active (like swapping RAM), are there even noises of retrying access? Even if not the video file, maybe a swap file...

Morku
10th May 2021, 09:02
Scanning for errors says, all fine:
https://i.imgur.com/5hFOwkK.png
Yes, the avs processing (unreleated to the encoding) just directly stop. There is no slowing down, warnings etc. ahead. The processing never finish, because it stops at a random frame count. There is also no more HDD activity (to read the avi) anymore. The onliest activity on this HDD is for Avisynth/Encoding. The HDD itself is still accessible and works fine when the issue occur. I can continue working normally on HDD. There are also no related errors in Windows eventviewer when the issue occur. It's only the avisynth processing which stops. My HDD Dock will bring the HDD even in sleep mode, because there is no more activity "during the progress".
But as said, I copied the video on my internal SSD with the same result, so this should be unreleated to the dock and used HDD.

Tried with ColorBars -> same issue.
Now I started to downgrade MVTools, MaskTools, RgTools, JPSDR tools one version by one. Tried 2 previous version with... same issue :|

real.finder
10th May 2021, 09:19
Morku, I note that you have avstp.dll, delete it and try

StainlessS
10th May 2021, 12:48
It would be a good idea to remove to a BackUp folder, all plugins not required by QTGMC, and retry on Colorbars.

If bug gone, continue using Colorbars, add back some dll's until problem returns, keep log of plugins added back, remove/addback
to narrow down problem plugin.

EDIT: Remove all plugins except those requred by QTGMC(Preset="Medium", EdiMode="NNEDI3", Sharpness=1.0)

Morku
10th May 2021, 13:21
I think real.finder found the issue =)
I removed avstp.dll and it's the first time AVSMeter finished the video.

So I updated my dll files back to latest, loaded my original Script and directly run a MeGUI encoding and it finished as well!
Thank you so much!

I didn't thought the reason could be found with that odd issue. Also it never occur before.

manolito
10th May 2021, 14:50
Interesting...

I never had a problem with avstp.dll, probably because I followed a recommendation by almosely.
https://forum.doom9.org/showthread.php?p=1865279#post1865279
The relevant information is in the third paragraph of the post.

At the end of any MT enabled script I have this:
avstp_set_threads(1)
Prefetch(Min(Int(Value(GetSystemEnv("NUMBER_OF_PROCESSORS"))),8))

The prefetch command was inspired by Myrsloik, and for my Core i5 (2 physical cores plus Hyperthreading) this results in a prefetch value of 4.

real.finder
10th May 2021, 17:24
I think real.finder found the issue =)
I removed avstp.dll and it's the first time AVSMeter finished the video.

So I updated my dll files back to latest, loaded my original Script and directly run a MeGUI encoding and it finished as well!
Thank you so much!

I didn't thought the reason could be found with that odd issue. Also it never occur before.

that mean it's mvtools bug with avstp.dll

Dogway
10th May 2021, 20:07
Is this by design?
expr(" sx width / range_size * ", "range_half","range_half", scale_inputs = "none" )
Creates a PC levels ramp.

ConvertBits(16)
expr(" sx width / range_size * ", "range_half","range_half", scale_inputs = "none" )
Creates a TV levels ramp.

EDIT: Might be avspmod related (https://forum.doom9.org/showthread.php?p=1942719#post1942719).

pinterf
11th May 2021, 12:51
that mean it's mvtools bug with avstp.dll

We know nothing about the problem other than removing avstp dll from behind a complex script stopped random freezing after some hundred thousand frames.

I suppose Morku is the only one with such experiences, and would like to help solving the issue :)

You can report it for me or for anyone but if something is not reproducible then it will be unanswered for sure.

- monitor processor temperature (though using avsmeter w/o encoding doesn't seem to have much burden on the processor)
- keeping avstp there in its place (which version?), then
- start providing mt=false parameters one-by-one for the appropriate mvtools functions (I don't know what is used in the actual script, probably MAnalyze, MDegrain, MSuper? which have such mt parameter which can be disabled. See mvtools html docs)
- Run qtgmc with parameters which do less and less things inside.
- or build a script from zero until you experience the freeze

Morku
11th May 2021, 17:24
I would like to help to solve the issue, but from current perspective I think it's a waste of time.
I copied back avstp.dll (1.0.4 x64 btw.) and guess what... I run the script in AVSMeter and it finished fine again. This makes it suspicious if removing avstp.dll really was the issue. But it was the onliest I changed on my system to solve it.

Also other utvideo avi with QTGMC and shorter runtime (a 20 minutes video, 30 minutes video, 10 minutes video) directly encoded fine before I had the trouble with that longer time video (90 minutes) which stopped processing mostly after 8-15 minutes of video.

If I could see in detail whats going on in background and 'whatever part of my PC stops for no reason'... it could be helpful.

Temperature during AVSMeter was about under 60°C on my CPU (HWiNFO).

If there is anything you might think I could test, please let me know, I will do, but you might need show me in detail what to change and load.
Now, I will keep avstp.dll just to see if the issue reoccur and if deleting avstp.dll will solve it again.
In 30 minutes is Patchday. Pretty sure this time Microsoft will solve all Windows 10 bugs ;)

mp3dom
11th May 2021, 23:44
I have the same issue here, and happens with plain QTGMC() (preset medium without any other parameter) and something more complex. On VDub I get the error that it's waiting the frame without success and ask me if I want to wait more. Obviously it stays there forever. If I abort the process then Vdub crash.

I actually have avstp but I'm not calling it in any way (it's not in the autoload folder, I have nothing in autoload besides the default 4-5 plugins installed by default by Avisynth).
I'll try on my side as well and report back (I'm using QTGMC intensively lately). Didn't report earlier because no one reported similar issue, I thought it was my fault or my configuration error...

It's very difficult to "reproduce" because sometimes it happens and sometimes not, even with the same file. Sometimes 1 time every 5, and sometimes 2-3 times every 5. It seems quite "random".

real.finder
12th May 2021, 04:09
I have the same issue here, and happens with plain QTGMC() (preset medium without any other parameter) and something more complex. On VDub I get the error that it's waiting the frame without success and ask me if I want to wait more. Obviously it stays there forever. If I abort the process then Vdub crash.

I actually have avstp but I'm not calling it in any way (it's not in the autoload folder, I have nothing in autoload besides the default 4-5 plugins installed by default by Avisynth).
I'll try on my side as well and report back (I'm using QTGMC intensively lately). Didn't report earlier because no one reported similar issue, I thought it was my fault or my configuration error...

It's very difficult to "reproduce" because sometimes it happens and sometimes not, even with the same file. Sometimes 1 time every 5, and sometimes 2-3 times every 5. It seems quite "random".

fun fact: even if you don't put avstp.dll in autoload folder, if you put it with mvtools or any other plugin that use it in same folder and then load that plugin in avs that plugin will load it!

mp3dom
14th May 2021, 10:50
Indeed, without avstp at all I've batch-processed more than 20 files (at 25min each on average) without any issue... and never encounter this success-rate before.
Avstp version used was 1.0.4 (the last compile from 2020/2021)

Morku
15th May 2021, 19:25
Good to know I am not alone with that odd issue and more importantly, I am not insane :)

real.finder
15th May 2021, 20:50
ok then, since QTGMC is huge, maybe start test with

super = MSuper()

bv2 = super.MAnalyse(isb = true, delta = 2, overlap= 4)
bv1 = super.MAnalyse(isb = true, delta = 1, overlap= 4)
fv1 = super.MAnalyse(isb = false, delta = 1, overlap= 4)
fv2 = super.MAnalyse(isb = false, delta = 2, overlap= 4)

MDegrain2(super, bv1, fv1, bv2, fv2, thSAD=300, thSADC=150)

sofakng
19th May 2021, 19:54
Does AviSynth (+) support DXVA2 or D3D11 hardware decoding?

I'm confused on how my system is playing back videos...

Here is my configuration:

- Windows 10 x64 (Nvidia GTX 3080 / Intel i7-8700K @ 5.0 GHz)
- LAV Filters v0.75
- AviSynth+v3.7.0
- AviSynth Filter (https://github.com/CrendKing/avisynth_filter) v1.0 (no scripts loaded)
- MPC-HC v1.9.8.75 (with AviSynth Filter enabled with 'Preferred' but no scripts loaded)

If I select 'D3D11' (native or copy-back) with LAV then I get a black screen.
If I select 'DXVA2' (native or copy-back) it works but is extremely slow with 7K or 8K video.

The only method I can successfully play 7K or 8K is using D3D11 (native) or DXVA2 (native).

1) Does AviSynth+ support D3D11 hardware decoding in LAV Filters?
2) Does AviSynth+ force DXVA2 (native) into copy-back mode? For example, my 7K/8K videos playback perfectly with DXVA2 (native) but adding AviSynth Filter (but no scripts!) causes the playback to become extremely slow similar to copy-back methods.

LigH
19th May 2021, 20:57
Decoding is not a matter of AviSynth, but a matter of the used source plugins. AviSynth is a video filter and frame server, optimized for accuracy over speed. It is not a replacement for DirectShow decoders.

sofakng
20th May 2021, 14:26
OK - I understand that AviSynth is not a decoder (ie. I'm using LAV Filters with hardware decoding), but AviSynth needs access to the decoded frame for processing, right?

Does this mean AviSynth is NOT compatible with DXVA2 or D3D11 Native-mode? Does it require copy-back instead?

LigH
21st May 2021, 07:29
There are decoder plugins which may have access to DirectX accelerated decoders, e.g. DirectShowSource, or to decoders in specific GPU chipsets, like DGDecNV using Nvidia PureVideo. But then, AviSynth needs the frame content in main RAM. A decoder which does not return the frame content but displays it immediately (i.e. connects to a renderer directly) has no value for a frame server which is supposed to filter the frame content further.

In most cases, AviSynth is used to serve filtered video to an encoder. In your case, it seems you try to use AviSynth to filter a video in a media player in real time?

subterrestrial
28th May 2021, 15:11
Thank you for so much effort on avisynth. It really helps me a lot.
I want to report an error "0xC0000005: Access violation reading location" which occured when I was using AVS+ 3.70 with CUDA option enabled and the aWarpsharpMT or the previous aWarpsharp2 plugin.
In Nekopanda's avisynth script, there is a line " SetDeviceOpt(DEV_CUDA_PINNED_HOST) ".
According to the comment, it's for CUDA data transport optimization. When adding this line to the script, the error occurs with using aWarpsharpMT/2. If without using aWarpsharpMT/2, everything goes fine even when I add that line. I will test the efficiency diffenrence with or without the line later and report here.
Test result:encoding 2000 frames, the same script with " SetDeviceOpt " was running about 20.5fps, without " SetDeviceOpt " was about 18.7fps.
When a function using aWarpsharpMT was added to the script without " SetDeviceOpt ", it run at about 4fps, if all functions in the script were switched to CPU version to make it run completely in CPU, the result of which was about 6fps.

Another problem is the KDeblock function in KFM project doesn't work, which needs a QP_table argument passed from AMTsource function in Amatsukaze project to work properly.
(Amatsukaze.dll is included in Nekopanda's encoding GUI called Amatsukaze which also in his github repository. He applied some nice ideas to his KDeblock function. According to his description, KDeblock uses the same algorithms as SmoothD does, but it is much flexible and adaptive in using a switchable deblock intensity parameter according to the QP value of each block of the source calculated and stored in QP_table by an indexer named AMTsource. So I wonder if somebody can integrate the QP_table calculation function to the KDeblock to make it work or maybe make amatsukaze compatible with the new avisynth header file. I tried to compile the Amatsukaze project with the new avisynth header file and do some modification . But I'm just a beginner to learn programming. That job is beyond my capability for now.)

GMJCZP
28th May 2021, 17:47
I am having a serious problem with MT and Prefetch, I am using the latest version of mtmodes.avsi (although it is irrelevant since using it or not the situation does not change). Here I publish the AvsMeter reports for Prefetch (1) and Prefetch (2):

Prefetch(1):
Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 86.51 | 185.8 | 129.7
Process memory usage (max): 37 MiB
Thread count: 7
CPU usage (average): 68.3%

Time (elapsed): 00:00:01.249


[Script]
LWLibavVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(1)


Prefetch(2):
Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 0.828 | 389652 | 8.301
Process memory usage (max): 41 MiB
Thread count: 8
CPU usage (average): 82.7%

Time (elapsed): 00:00:19.515


[Script]
LWLibavVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(2)

Here Sample2 (http://www.filedropper.com/sample2)

I am using last version of Lsmash.

Interestingly, when I compile the script the time difference is very little.

DorianDiaconu
30th May 2021, 22:39
Hi bois,

Does it work on Mac, Fedora and other linux distros? Can I install it with wine?

Thanks!

pinterf
31st May 2021, 08:54
Thank you for so much effort on avisynth. It really helps me a lot.
I want to report an error "0xC0000005: Access violation reading location" which occured when I was using AVS+ 3.70 with CUDA option enabled and the aWarpsharpMT or the previous aWarpsharp2 plugin.
In Nekopanda's avisynth script, there is a line " SetDeviceOpt(DEV_CUDA_PINNED_HOST) ".
According to the comment, it's for CUDA data transport optimization. When adding this line to the script, the error occurs with using aWarpsharpMT/2. If without using aWarpsharpMT/2, everything goes fine even when I add that line. I will test the efficiency diffenrence with or without the line later and report here.

I'd need that script or better: if you could provide me a minimalistic script that still exhibits Access Violation.
Some words on the status of Avisynth+ and CUDA build: I was able to backport Nekopanda's changes and adjust the plugin set.
Translated the documentation and documented some of my (beginner's) experiences.
I even bought an NVidia card for future experiments. I am happy when someone (other than me :) ) is finally testing whether these backports are working or not; unfortunately I can start fixing something that I am able to reproduce.

As for the existing filters that were fitted to Neo's Avisynth headers and frame property access, I think I cannot help with them. The filters I ported required three main tasks to do: changing Avisynth+ headers, change some names in IScriptEnvironment calls, and reimplement code parts accessing frame properties - this latter is very much different.

FranceBB
31st May 2021, 10:55
Hold on, I wanna test CUDA filters too! xD
If they work, I guess I'm gonna be in for a treat 'cause they would speed things up a lot.
What filters do we have and what should I do to make them work with CUDA?

And is everything that works internally cuda accelerated now?

Like:

ColorBars(848, 480, pixel_type="Yv12")

OnCuda()

tweak(sat=1.31, dither=true)

Spline64Resize(1280, 720)

Do I need a special build of Avisynth+?

I mean, afaik everything from the Nekopanda version has been ported, right?
Is there any kind of documentation for our normal Avisynth+ with cuda so that I don't have to use Google Translate to translate from Japanese to English? xD

pinterf
31st May 2021, 11:37
Hold on, I wanna test CUDA filters too! xD
If they work, I guess I'm gonna be in for a treat 'cause they would speed things up a lot.
What filters do we have and what should I do to make them work with CUDA?

And is everything that works internally cuda accelerated now?

Like:

ColorBars(848, 480, pixel_type="Yv12")

OnCuda()

tweak(sat=1.31, dither=true)

Spline64Resize(1280, 720)

Do I need a special build of Avisynth+?

I mean, afaik everything from the Nekopanda version has been ported, right?
Is there any kind of documentation for our normal Avisynth+ with cuda so that I don't have to use Google Translate to translate from Japanese to English? xD
CUDA-capable Avisynth+ is simply an Avisynth+ which can support CUDA-aware filters. Nothing is accelerated inside.
Due to the complexity - special knowledge, higher maintenance cost (=time, additional testing) and in general supporting only a subset of the parameters - it will probably never appear in the core.
You can use mt_lut from (a modified branch) Masktools2, KNNEDI3, Merge, Resizers, etc. from Avisynth core as an external plugin. This was done by Nekopanda and have to be built against the actual Avisynth headers.
The 3.7.0 release for example is not yet capable for accepting CUDA filters due to some leftovers at release-time which were fixed since then.

CUDA-aware Avisynth+ build is an additional option in CMake build process, I'm using CMakeGUI, there is a simple checkbox for that. When it finds up-to-date CUDA SDK, you can build your version as well. If I remember correctly, the first post in this topic links to a pack with a CUDA-aware build inside.

Again: if you have no CUDA capable filters it will do the same as an ordinary Avisynth+ does. I recommend reading the documentations at
https://github.com/pinterf/AviSynthCUDAFilters/tree/master/documentation

kedautinh12
31st May 2021, 11:39
Here had built x64 CUDA
https://drive.google.com/uc?export=download&id=1CpFdkqbNRDwtuHCtWiQ4x49ZCt8W2jkS

subterrestrial
31st May 2021, 12:40
Thanks for replying and explaining.
I use CUDA plugins to do deinterlace, MPEG2 bad field cleaning (DecombUCF), deblock, deband. To make things simple and clear, I will just write a simple script to produce the access violation error using SetDeviceOpt(DEV_CUDA_PINNED_HOST) statement and awarpsharp2/MT plugin without any CUDA plugin.
<code>
LWLibavVideoSource("L:\....mkv")
SetDeviceOpt(DEV_CUDA_PINNED_HOST)
aWarpSharp2(depth=32,blur=2,chroma=3)
</code>
I'm not sure whether the problem is caused by my compilation or not. If so, please pardon me for being amateur.
Now I guess I understand the difficulties to make Kdeblock work, so there is less wonder why Nekopanda seperate the Kdeblock code both in KFM and an indexer if QP_table is associated with frame property.
The idea of Kdeblock is somewhat like jconklin's amDCT using adaptive deblock strength. The former one lacks x64 version and need heavy computing like DCT and iDCT which I was told GPU is good at. So maybe in the future these methods might be inplemented and functioned.
Anyway, I'm really satisfied with CUDA plugins now. I can't tell the difference of the result of KTGMC and conditional QTGMC both using slow preset. Without your effort, maybe I have to switch between avisynth+ and avisynthneo if encountering compatible problems.

subterrestrial
31st May 2021, 13:24
The Nekopanda's AvisynthNeo and plugins using CUDA are components of his highly automated video processing and encoding GUI "Amatsukaze" which can do ADs cutting, delogo, video process, subtiles extracting, and merging and muxing all the things together only need you to set the parameters, drag the file in and leave it till finish.
My guess is he created all these AvisynthNeo stuff for his GUI working with high efficiency.
You can find all these in his Github repository "github.com/nekopanda". Printerf already translated his Avisynth documents in English, but I'm afraid you have to use google to translate documents other than that in his repository.
I think you are experienced in writing script for processing video, maybe you don't need further examples to show how to use these functions. But in case you need you can download the Amatsukaze software, in the Amatsukaze\avs path there is a compressed file named “_旧バージョンのavsファイル”,it stored some example avs files to deal with different kinds of source files.

GMJCZP
31st May 2021, 14:18
I am having a serious problem with MT and Prefetch, I am using the latest version of mtmodes.avsi (although it is irrelevant since using it or not the situation does not change). Here I publish the AvsMeter reports for Prefetch (1) and Prefetch (2):

Prefetch(1):
Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 86.51 | 185.8 | 129.7
Process memory usage (max): 37 MiB
Thread count: 7
CPU usage (average): 68.3%

Time (elapsed): 00:00:01.249


[Script]
LWLibavVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(1)


Prefetch(2):
Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 0.828 | 389652 | 8.301
Process memory usage (max): 41 MiB
Thread count: 8
CPU usage (average): 82.7%

Time (elapsed): 00:00:19.515


[Script]
LWLibavVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(2)

Here Sample2 (http://www.filedropper.com/sample2)

I am using last version of Lsmash.

Interestingly, when I compile the script the time difference is very little.

Friends, I am working with one core, It is unacceptable.

pinterf
31st May 2021, 15:41
This is even slower:
LWLibavVideoSource("Sample2.mp4", threads = 1)
Prefetch(2)
Seems that LWLibavVideoSource does not like out of order frame requests (I suppose we have a such scenario). To my eyes it looked like it was slowing down in a linear way as if it would always start decoding from the very first frame whatever frame number is requested.

The frames parameter could help a bit though.
Prefetch(2, frames=2)

Looked at FFMS2 but it didn't show such problems.

FranceBB
31st May 2021, 17:16
CUDA-capable Avisynth+ is simply an Avisynth+ which can support CUDA-aware filters. Nothing is accelerated inside.
Due to the complexity - special knowledge, higher maintenance cost (=time, additional testing) and in general supporting only a subset of the parameters - it will probably never appear in the core.
You can use mt_lut from (a modified branch) Masktools2, KNNEDI3, Merge, Resizers, etc. from Avisynth core as an external plugin.
The 3.7.0 release for example is not yet capable for accepting CUDA filters due to some leftovers at release-time which were fixed since then.
https://github.com/pinterf/AviSynthCUDAFilters/tree/master/documentation

Got it, thanks for the explanation and thanks for the link to the English documentation! :D

Here had built x64 CUDA
https://drive.google.com/uc?export=download&id=1CpFdkqbNRDwtuHCtWiQ4x49ZCt8W2jkS

Great! Thanks! ;)

GMJCZP
31st May 2021, 17:28
Thanks pinterf.

With frames = 2 it improved noticeably but still doesn't even match Prefetch 1:

Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 0.971 | 107491 | 29.03
Process memory usage (max): 37 MiB
Thread count: 8
CPU usage (average): 64.9%

Time (elapsed): 00:00:05.581


[Script]

LWLibavVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(2,frames=2)

I include the test with FFMS2:

Log file created with: AVSMeter 3.0.9.0 (x86)
Script file: Prueba.avs
Command line switches: -log

[OS/Hardware info]
Operating system: Windows 7 (x86) Service Pack 1.0 (Build 7601)

CPU: Pentium(R) Dual-Core CPU E5800 @ 3.20GHz / Wolfdale (Core 2 Duo) 2M
MMX, SSE, SSE2, SSE3, SSSE3
2 physical cores / 2 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.0 (r3382, 3.7, i386)
VersionNumber: 2.60
File / Product version: 3.7.0.0 / 3.7.0.0
Interface Version: 8
Multi-threading support: Yes
Avisynth.dll location: C:\Windows\system32\avisynth.dll
Avisynth.dll time stamp: 2021-01-11, 20:46:40 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files\AviSynth+\plugins+


[Clip info]
Number of frames: 162
Length (hh:mm:ss.ms): 00:00:06.757
Frame width: 640
Frame height: 480
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P16
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 162 (0 - 161)
FPS (min | max | average): 0.756 | 239787 | 5.697
Process memory usage (max): 35 MiB
Thread count: 7
CPU usage (average): 81.5%

Time (elapsed): 00:00:28.437


[Script]
FFVideoSource("Sample2.mp4")
AssumeFPS("ntsc_film")
Prefetch(2)

qyot27
31st May 2021, 17:48
Hi bois,

Does it work on Mac, Fedora and other linux distros? Can I install it with wine?

Thanks!
http://i.imgur.com/OCNAx9Kh.jpg (https://imgur.com/OCNAx9K)

http://i.imgur.com/Jzl6qPlh.png (https://imgur.com/Jzl6qPl)

https://github.com/AviSynth/AviSynthPlus/blob/master/distrib/docs/english/source/avisynthdoc/contributing/posix.rst

https://repology.org/project/avisynthplus/versions

Also see the thread topics in my signature.

jpsdr
31st May 2021, 17:48
I want to report an error "0xC0000005: Access violation reading location" which occured when I was using AVS+ 3.70 with CUDA option enabled and the aWarpsharpMT or the previous aWarpsharp2 plugin.

At least, it's not specific to my version...
But this realy looks like an unaligned access....

pinterf
31st May 2021, 20:48
At least, it's not specific to my version...
But this realy looks like an unaligned access....
My script:
SetDeviceOpt(DEV_CUDA_PINNED_HOST)
LoadPlugin("c:\github\aWarpSharpMT\x64\Debug\aWarpSharpMT.dll")
Colorbars(pixel_type = "YUV420P16")
aWarpSharp2(depth=32,blur=2,chroma=3)

Not specific. In the actual version the crash is here (and probably in the other processor/MT implementations as well):
https://github.com/jpsdr/aWarpSharpMT/blob/master/aWarpSharpMT/aWarpSharp_asm_x64.asm#L1327

Trying to read before the leftmost column in Sobel. It must be an old bug, inherited from the the beginnings.

EDIT:
Ha! What have I got in my pocket :) I found it in my earlier aWarpSharp repo:
https://github.com/pinterf/aWarpSharp/blob/master/src/aWarpSharp.cpp#L115
and
https://github.com/pinterf/aWarpSharp/blob/master/src/aWarpSharp.cpp#L120

My comment: // PF 170926 read before the very first byte! Dangerous. Todo eliminate.

I either had no time or the knowledge to do that four years ago.

It seems that DEV_CUDA_PINNED_HOST option is allocating the frame buffer memory area more strictly than the usual Avisynth allocation, thus the bug emerged.

DorianDiaconu
31st May 2021, 22:47
https://github.com/AviSynth/AviSynthPlus/blob/master/distrib/docs/english/source/avisynthdoc/contributing/posix.
[url]https://repology.org/project/avisynthplus/versions

Also see the thread topics in my signature.

Thanks a bunch!

I found those threads today after reading a bit. What software do you pair it on Linux with? On Windows I use it with Vdub.

Also I saw compatible with BSD and Ubuntu. What about Fedora?

You were of great help!

wonkey_monkey
31st May 2021, 23:17
The following is valid, but (in my opinion) shouldn't be:


x = source.trim(10,20)flipvertical


(note the lack of a "." between ")" and flipvertical - the call to flipvertical is ignored)

Similarly:


x = clip1 clip2


which has a missing "+" between the clips, which results in the "clip2" part being ignored.

Can/should such things cause errors, or is there a reason that can't happen?

StainlessS
1st June 2021, 04:03
Thats just
Last = Last.flipvertical
and
Last = clip2

(the final parts)

wonkey_monkey
1st June 2021, 10:05
Seems a bit silly to allow, though, especially when we're forced to use \ when we want to split a statement to multiple lines. In light of the above it seems like that should be unnecessary.

If


clip1 = source.flipvertical clip2 = source.fliphorizontal stackhorizontal(clip1, clip2)


is equivalent to


clip1 = source.flipvertical
clip2 = source.fliphorizontal
stackhorizontal(clip1, clip2)


(which it is) then I'm not sure why the parser couldn't just ignore newlines entirely (except for terminating comments) and let us do


clip1 =
flipvertical
clip2 =
fliphorizontal

stackhorizontal(
clip1,
clip2
)


without having to use \ (Also more languages these days allow a trailing "," in lists which is really nice :) )