Port of NNEDI under new v2.6 AVS API

[jpsdr]

Argh....:confused:
Virtualdub provides disassembled code where crash occured, which could have told me on what part was the crash. But without this...
In that case, i must sadly confess that i have no clue of what to do... :(

Does crash also happen with 32 bit version (under vmware) ?

Edit : And on the 13.1MB, 12.9MB are the neural network datas.;)

[jpsdr]

@stax76
I'm using the last stable release of AVS+, r1576. Do you use this version ? If not, can you test with r1576 ?

[stax76]

I did test r1576 and also Win10 natively (without vmware), problem persist.

[jpsdr]

I don't have time these days, but later, i'll check the ASM code in case in see something.
Can you try the "old" Tritical's build just out of curiosity : http://forum.doom9.org/showthread.php?t=147695 ?

[stax76]

Can you try the "old" Tritical's build just out of curiosity

Same result, without opt=1 mpc-be x64 crashes on Win10.

[jpsdr]

Are you using the "vectorcall_" instead of standard "fastcall_" for compiling your StaxRip 64b port ?

[stax76]

I use VB.NET and interface with AviSynth using the avifile API, only thing I saw was win32exception, I tried to attach a debugger but it didn't yield to anything, in the system log I found some info:

Fehlerbucket 86074866829, Typ 4
Ereignisname: APPCRASH
Antwort: Nicht verfügbar
CAB-Datei-ID: 85948023432

Problemsignatur:
P1: StaxRip.exe
P2: 1.3.1.1
P3: 554a041b
P4: nnedi3.dll
P5: 0.9.4.7
P6: 550421cb
P7: c0000005
P8: 000000000000fede
P9:
P10:

Angefügte Dateien:
C:\Users\frank\AppData\Local\Temp\WERFA59.tmp.WERInternalMetadata.xml
C:\Users\frank\AppData\Local\Temp\WER622.tmp.appcompat.txt
C:\Users\frank\AppData\Local\Temp\WER6CF.tmp.hdmp
C:\Users\frank\AppData\Local\Temp\WER2535.tmp.WERDataCollectionFailure.txt

Diese Dateien befinden sich möglicherweise hier:
C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_StaxRip.exe_b72abc897b1e0ecd8d22d975322fdbcbd313f93_a07971f2_cab_20277c46

Analysesymbol:
Es wird erneut nach einer Lösung gesucht: 0
Berichts-ID: d0f7bfad-401a-4a32-9ee9-89b9afdcde02
Berichtstatus: 8
Bucket mit Hash: d59ed2b9785e003d6467e1e13724dc70

Name der fehlerhaften Anwendung: StaxRip.exe, Version: 1.3.1.1, Zeitstempel: 0x554a041b
Name des fehlerhaften Moduls: nnedi3.dll, Version: 0.9.4.7, Zeitstempel: 0x550421cb
Ausnahmecode: 0xc0000005
Fehleroffset: 0x000000000000fede
ID des fehlerhaften Prozesses: 0x21ec
Startzeit der fehlerhaften Anwendung: 0x01d087f647effd06
Pfad der fehlerhaften Anwendung: D:\Projekte\GitHub\staxrip\bin\StaxRip.exe
Pfad des fehlerhaften Moduls: D:\Projekte\GitHub\staxrip\bin\Tools\Plugins\nnedi3\nnedi3.dll
Berichtskennung: 429aaf55-ee38-40c4-aec7-6dc86532b124
Vollständiger Name des fehlerhaften Pakets:
Anwendungs-ID, die relativ zum fehlerhaften Paket ist:

[jpsdr]

Unfortunately, i'm not able to get informations from this. I'm concerned by the fact that also the old Tritical's version is crashing. It's realy more troublesome, meaning that maybe issue is in the ASM code itself, and not caused by an error i could have done during the port.
You have no issues with the 32 bit version ?
And VDub working perfectly fine...!!
I think in two weeks i may have time to recheck the ASM code, in case there is errors in my port.
But at first glance, it's not looking good... :(
VDub works fine, old Tritical's version also crashing... bad... very bad...

[stax76]

I use x64 exclusively everywhere so it's difficult for me to test. I would have to download and setup everything. Is there a big performance difference between c and asm path? QTGMC uses nnedi3 and still works.

[jpsdr]

I don't have benchmark, and can't do for now, my video dedicated PC is encoding, and there is at least one day left. But as i remember when i've done test, performance issue is noticeable i think just sliding a file with VDub (or is it disabling internal mutli-threading... i have doubts finaly...) with an avs script with only one line using only nnedi3_rpow2.

[Groucho2004]

I did test r1576 and also Win10 natively (without vmware), problem persist.
I already asked you over at Don's forum - What client programs do you feed the script to? I don't think MPC-BE is a good choice for testing.
If the script works with VDub64, x264_64, AVSMeter64, I'd say your problem is not the plugin.

[videoh]

If the script works with VDub64, x264_64, AVSMeter64, I'd say your problem is not the plugin. + 1 Post of the day.

[stax76]

Just because a bug does not surface doesn't mean that the bug does not exist, the only thing that it sure at this point is that the bug exists, it can be in the avs plugin, it can be in the avs core, it can be in the client interop layer, it can be even in the CLR. I tested 15 plugins and only nnedi3, DGMPGDec and DGDecNV have a problem, chances are good we will find it, wherever it is.

[Groucho2004]

Just because a bug does not surface doesn't mean that the bug does not exist
I can't quite imagine how the bug you reported (crash) can be concealed by other programs.

DGMPGDec
When you search the forum, you'll find that there are all sorts of problems with the 64 Bit version of DGDecode.dll, missing IDCT modes, "green" frames, ...

[stax76]

When MediaInfo was new it wasn't working with any .NET application, I don't know what it was, it could be a bad pointer pointing into managed space I believe.

[videoh]

The 64-bit version of DGMPGDec's DGDecode is not made by Graft and is known to be buggy, as Groucho2004 pointed out. There are no known issues with 64-bit DGDecNV, so please demonstrate one if you can. Graft is always keen to fix issues in his code.

None of Graft's tools contain any "pointers pointing into managed space".

[jpsdr]

If the script works with VDub64, x264_64, AVSMeter64, I'd say your problem is not the plugin.
I personnaly would said "It may not be the pluggin, or don't focus only on pluggin".

Just because a bug does not surface doesn't mean that the bug does not exist.
I also agree.

The fact also that's it's not crashing with opt=1 means that it's crashing only when using the ASM part code, meaning that something broke the register presevation rules.
If it was a bug in the ASM like a loop going too far or something like that, issue would have poped up earlier and on any system (i think).
This is why within a week or two, i'll take a long time to properly check that the ASM code is not breaking the x64 register preservation rules, according the standard fastcall_ calling procedures. I'll also check code itself, but this is the only thing i think for now.

stax76
You've asked benchmarks :

[General info]
Log file created with: AVSMeter 2.0.1 (x64)
Avisynth version: AviSynth+ 0.1 (r1576, x64) (2.6.0.5)
Avisynth DLL location: C:\Windows\System32\AviSynth.dll


[Clip info]
Number of frames: 4077
Length (hh:mm:ss.ms): 00:02:50.045
Frame width: 960
Frame height: 720
Framerate: 23.976 (24000/1001)
Colorspace: YV12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 4077 (0 - 4076)
FPS (min | max | average): 17.08 | 99.95 | 55.80
Memory usage (phys | virt): 93 | 91 MB
Thread count: 37
CPU usage (average): 38%
Time (elapsed): 00:01:13.060


[Script]
SetMemoryMax(64)
AVISource("Ken - 101.avi",False,"YV12").SetPlanarLegacyAlignment(True)
nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",fwidth=960,fheight=720,nsize=0,nns=3,qual=2)




[General info]
Log file created with: AVSMeter 2.0.1 (x64)
Avisynth version: AviSynth+ 0.1 (r1576, x64) (2.6.0.5)
Avisynth DLL location: C:\Windows\System32\AviSynth.dll


[Clip info]
Number of frames: 4077
Length (hh:mm:ss.ms): 00:02:50.045
Frame width: 960
Frame height: 720
Framerate: 23.976 (24000/1001)
Colorspace: YV12
Audio channels: n/a
Audio bits/sample: n/a
Audio sample rate: n/a
Audio samples: n/a


[Runtime info]
Frames processed: 1903 (0 - 1902)
FPS (min | max | average): 15.32 | 36.55 | 26.79
Memory usage (phys | virt): 92 | 91 MB
Thread count: 37
CPU usage (average): 74%
Time (elapsed): 00:01:11.047


[Script]
SetMemoryMax(64)
AVISource("Ken - 101.avi",False,"YV12").SetPlanarLegacyAlignment(True)
nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",fwidth=960,fheight=720,nsize=0,nns=3,qual=2,opt=1)

Input video is encoded with UT Video codec.

[foxyshadis]

Just because a bug does not surface doesn't mean that the bug does not exist, the only thing that it sure at this point is that the bug exists, it can be in the avs plugin, it can be in the avs core, it can be in the client interop layer, it can be even in the CLR. I tested 15 plugins and only nnedi3, DGMPGDec and DGDecNV have a problem, chances are good we will find it, wherever it is.

Can you capture a crash dump in WinDbg, or install VS on the VM to debug it realtime? That might help a lot more. Access Violation on its own doesn't really tell a lot.

[stax76]

I already migrated to Win10 and Visual Studio 2015 Community Edition RC, I can try things but would need some hints because my C++ knowledge is only very basic.

[jackoneill]

I use x64 exclusively everywhere so it's difficult for me to test. I would have to download and setup everything. Is there a big performance difference between c and asm path? QTGMC uses nnedi3 and still works.

The ASM is something like 10 times faster, if I remember correctly (with default parameters).

jpsdr: I noticed this while perusing your ASM: https://github.com/jpsdr/NNEDI3/blob/master/nnedi3/nnedi3_asm_x64.asm#L1414 I don't know if that function is even used when it crashes, since stax76 hasn't provided the parameters passed to nnedi3... :(

[jpsdr]

Good !!! And :thanks: I don't know if it's related or the cause (there a good chance), but it's clearly a miss from my part wich need to be corrected.
Even with stax76 parameters, i don't know if i can say if this function is called or not.
I'll try to make a new build as soon as i can.

[jpsdr]

New buid avaible for test. :cool:

[stax76]

It still crashes, I'm trying to build the solution now with VS 2015 RC, it says /ZI and /O2 are incompatible.

[Groucho2004]

it says /ZI and /O2 are incompatible.
Optimizations have to be turned off for debug builds.

[stax76]

Thanks, I turned off optimizations and replaced

this line:

#include "afxres.h"

with:

#include "WinResrc.h"
#define IDC_STATIC -1

I thought MFC is included with Community Edition?

The build succeeds now but why doesn't it output any DLLs?

[stax76]

It's always breaking as access violation in this line:

https://github.com/jpsdr/NNEDI3/blob/19a21726b8747cd691b33297b5e24615d55ab86d/nnedi3/nnedi3.cpp#L1258

There is however something in my code because I just noticed it only happens in the preview dialog and it loads in the crop dialog fine, I didn't expect that. I don't remember I do threading or something special. At least we have some hints now.

edit:

crop dialog does not use any other filters than the source I was forgetting

[Groucho2004]

It's always breaking as access violation in this line:

https://github.com/jpsdr/NNEDI3/blob/19a21726b8747cd691b33297b5e24615d55ab86d/nnedi3/nnedi3.cpp#L1258

There is however something in my code because I just noticed it only happens in the preview dialog and it loads in the crop dialog fine, I didn't expect that. I don't remember I do threading or something special. At least we have some hints now.

edit:

crop dialog does not use any other filters than the source I was forgetting
Have you tried "threads=1" with nnedi?

[stax76]

Have you tried "threads=1" with nnedi?

does not make a difference, it works fine with opt=1

[jpsdr]

Picture information is interesting. If registers value are correct, RAX & RCX at 0, crash is expected. But issue is that RAX shouldn't be at 0, it's the source pointer. As this crash occur during the y loop, is the crash occuring at the first occurence of the loop, or after several ?

[jpsdr]

I think i found another miss from my part, i'll do another build ASAP (but stay with same version).

[jpsdr]

New build avaible. With a little luck, maybe the good one...:rolleyes:

[stax76]

You made it, this is a great day for AviSynth x64, for StaxRip x64 and for people using different languages working together, it has been a pleasure to work with you on this, thanks! :thanks:

[jpsdr]

Wonderful. It's also a good thing to get ridance of lefting bugs. The only surprising thing, it's that these bugs left could have been trigged by anything. It was clearly not a specific issue to Win8.1/Win10. Well, everything ends well, it's what matters.
Thanks also at the others who have helped, providing information and running debug builds.

[stax76]

@jpsdr

DGDecNV and mvtools2 behave the same as nnedi3 before you fixed it. They work in VirtualDub but cause a access violation and crash in managed applications and in MPC-BE. Is there anything I can tell the authors how to find the problem? Did the line I showed you help/relate?

[jpsdr]

Unfortunately i don't think so. My issue was clealy a bug in the ASM file, some registers where kept with their 32bit versions when they should have been their 64bits versions. What helped me more (and that was what i needed) was the screenshot of HolyWu with disassembled part of the crash, and the status of the registers when crash occured.

[videoh]

They work in VirtualDub but cause a access violation and crash in managed applications Let's be precise: they crash in your managed applicaton. There is no problem in DGDecNV. And the threads here show that you refuse to answer questions or perform any straightforward debugging to actually try to locate the problem. I stand ready to assist if you are actually interested in solving the problem, but it seems that you are more interested in insulting people because they do not use .NET. That is my reward for trying to help you.

[stax76]

I told you about 15 times that all you need is loading a DGDecNV avs script with MPC-BE x64 or StaxRip x64 and you will see a access violation happening in YOUR native code, you were wrong, accept it, let it go, fix your code.

[videoh]

I don't have the environment setup for opening avs scripts in the DirectShow-based players (they can't open AVS on my system and I never figured out why), so I can't currently test MPC-BE. Can you please show me the crash screen?

[burfadel]

I see there is someone doing active work on NNEDI3 for Vapoursynth. Are any of those commits compatible with NNEDI3 for Avisynth, and if so can they be ported if they haven't already been done so?

https://github.com/dubhater/vapoursynth-nnedi3/commits/master

[jpsdr]

Projects are too different, and so very very unlikely.

[burfadel]

Fair enough :). I just wasn't sure to what extent that the two were different.

[Myrsloik]

Projects are too different, and so very very unlikely.

I think this commit may be of interest to you:
https://github.com/dubhater/vapoursynth-nnedi3/commit/89819db62fafcc147fa5bc9d02c20812b761cfee

Equivalent line probably here:
https://github.com/jpsdr/NNEDI3/blob/master/nnedi3/nnedi3.cpp#L471

Not sure if this commit is also relevant but maybe it should be checked:
https://github.com/dubhater/vapoursynth-nnedi3/commit/8e97772face212a950f04380581bd554d4ec00d4

[jpsdr]

Thanks, i'll take a look.

[jpsdr]

New version avaible.

[burfadel]

New version avaible.

Thanks :)

[hello_hello]

If anyone cares, the right click explorer file/properties menu shows version 0.9.4.10 of the various nnedi3.dlls as being version 0.9.4.7. It was the same for 0.9.4.9.

Cheers.

[jpsdr]

Ah... Thanks for the information, i totaly forgot this in the compiler settings. And i have to find out where it was again... I'll do a new build very soon, even if not critical, it's always usefull.

[jpsdr]

New build avaible with correct version in properties file.

[jpsdr]

If anyone have any idea of how correct nnedi3 according the informations here :
http://forum.doom9.org/showthread.php?p=1660744#post1660744

I personnaly don't know how to "fix" the actual code.
I know the opencl version is corrected, but code is too much different, i don't know how to implement it in the actual code.

I don't know if the Vapoursynth version is "fixed".

Any information or even pull request are welcomed. :D

[jpsdr]

New version, i've tried to fix the chroma shift according the informations on this thread :
http://forum.doom9.org/showthread.php?t=170029

Test and report any issue, any feedback is welcomed.