SEt's Avisynth 2.5.8 MT compiled for *X86_64*, Latest Build 4/16/2010

[osgZach]

Also it would be really nice if we could get an updated DGDecode DLL based on 1.5.8

I've gone and gotten DGIndex 1.4.6 in the meantime, and I'm hoping there is going to be no major differences in the results? Though I'm guessing I should recollect my YATTA metrics just to make sure..

[JoshyD]

Thanks for your response.

Those are certainly some nice numbers. I wasn't even aware you could run it multi-threaded either. Frankly the setup process for MT related stuff scared me way, was afraid I would boink something :p
Right now I only exclusively use TempGaussMC_beta2, the only other filter being TDecimate, and any other stuff YATTA deems necesarry in the generated AVS. I am encoding to HuffYV12 and then filtering later. So hey maybe I'll get something like 5 or 6 fps ? LOL :p


Perhaps I will go back to the initial post and see if I can follow the steps to do all of this..

Although as far as MT goes.. I only have 2 cores, so I wouldn't expect huge numbers like you got (but really impressive), but I think the x64 single threaded boost might be pretty big as well.

Is there a chance this will all be available vial a one-click installer at some point?

There is a one click installer included in the package, it was written by another user, turbojet. You just extract the files to any directory, and run it.

It needs to be run as administrator, and should copy all the needed info to run 64 bit mode.

Any hiccups, come back, and definitely ask. I really appreciate user feedback, it helps me understand where I can make the project more user friendly, and what makes people decide not to try it.

If that doesn't work, try this as a fallback:
Essentially, you need to put Devil.dll and avisynth.dll in your system32 directory. All other plugins can be loaded in your script or placed in <Avisynthdir>\plugins, or I think the installer changes it so that 64bit plugins go in <Avisynthdir>\plugins64.

There should be two system directories under <drive>:\Windows, one for SysWOW64, and one for system32. System32 is a misnomer, as this is where all 64bit binaries are stored. SysWOW64 stands for system windows on windows 64, and is meant to keep 32bit binaries separate from your 64bit ones. SysWOW64 is where your current avisynth.dll lives.

Other notes, SetMTMode is really worth getting to know a little better. If you have two cores, why not use them? I'll be happy to help you get started, just send a quick message if interested.

[osgZach]

I'd definitely like to learn how to use SetMTMode.. I think there are some threads around here I might be able to read to get a handle on it, too. But I'll try not to ask anything -too- stupid if it comes down to needing help.

A quick example with TGMC would be best, as that's all I really need at this point.

I did use the batch install script. It worked for the most part. Was it supposed to make a "C:\Program 2.5\plugin64" directory though? I am stuck with it now, and it will only autoload plugins from that directory. It's a minor issue at this point though, I probably goofed on setup procedure/steps.

My only remaining problem is.. I need Decimate or TDecimate , to do my IVTC. I did have to hack around the AVS that YATTA generated, to get things working (until I discovered only that aforemention directory was the one needed for auto-loading) so far I get it opened to the point where it whined about no TDecimate :p

I'm guessing there is no x64 Tdecimate as its part of TIVTC package =\

Is there a Decomb 64-bit package? Otherwise I'm kind of... meh now. can't proceed any further until I figure out a solution. Hoping the answer is in front of my face, lol....:(

[JoshyD]

So, are we currently not able to use TIVTC and most of Tritical's plugins with the 64 bit version of Avisynth? Or is that just because they haven't been converted yet?

If so, I request that we convert as many of Tritical's plugins as possible. Specifically TIVTC and Colormatrix, as I think that those are some of the most often used plugins.

TIVTC is going to be an undertaking, but it's been so largely requested, it seems that I may have to suck it up and do it.

ColorMatrix looks easier, I may do that one today . . .

[JoshyD]

Ta-da! Decomb64 (http://members.optusnet.com.au/squid_80/Decomb.zip) This was built by squid_80 way back when. If it doesn't give you the desired results, I'll look into building a version myself. For more quick existing ports check his page out. (http://members.optusnet.com.au/squid_80/)

There isn't a TDecimate that I'm aware of, but, TIVTC is getting ported one way or another, dammit.

[Stephen R. Savage]

I believe the poster who mentioned Decomb wanted the v4 release, not the v5 built by Squid80 (which does work fine). However, I think porting TIVTC would make his point moot.

[osgZach]

Actually Stephen, up until now, I have always used Decomb V5, but will switch over to TFM/TDec when they are finally ported to x64.

However, since I only need the Decimate function (at least I can still index with TIVTC in 32-bit) and not Telecide or anything else, I think this will work for me.

Pretty much going to be going TFM/TDec in the foreseeable future. Works for my needs from what I can tell.

I did start an AVI (Lagarith since I don't have ffdshowx64 installed) I saw a slight improvement but really not much.. I don't seem to dip into the 1.x FPS range anymore, and on average its about 2.5fps or so. But I am hoping once I get MT figured out, I will see a better performance boost.

In either case, instead of just hitting every frame with TGMC, I can just set my TFM values really low to pick up all visible combing until I'm satisfied. That'll give me a good performance boost. ON default values, earlier today I did a test and it only took 30m to do the encode to HuffYV12. So almost real-time results which is not bad.

I'm probably abusing TGMC anyway, but it gives such good quality whether the frame is actually combed or not, I figure why not just wipe it all out in one fell swoop :p

If I could figure out that fuzzy math behind VFR trimming + timecodes frame-range calculations, and actually manage to write it in code, I wouldn't even need YATTA for TFM stats... ah well, its a great program regardless. And so is Avisynth x64 (I hope!)


UPDATE

Nice! SetMTMode(2,0) gets me between 3 - 4 fps now instead of 2 on average. It goes up and down. Bit of a boost tho :) I've even seen it go up to 5fps for a few seconds.

[osgZach]

OK that's not good.... It just threw an Access Violation. It was somewhere around 80 - 90% done with the encode I think.

Running SetMTMode(2,0) outputting to Lagarith, YV12, Multithreaded, and at the time of the crash it had 749MB of memory allocated in the processes tab.


Crash context:
An out-of-bounds memory access (access violation) occurred in module 'Veedub64'...

...writing address 0000000000000000.


Crash dump included below.


VirtualDub crash report -- build 32706 (release-AMD64)
--------------------------------------

Disassembly:
7fef045b100: 49 dec ecx
7fef045b101: 03d3 add edx, ebx
7fef045b103: 49 dec ecx
7fef045b104: 03c8 add ecx, eax
7fef045b106: 49 dec ecx
7fef045b107: 03d0 add edx, eax
7fef045b109: 43 inc ebx
7fef045b10a: ff24c2 jmp dword ptr [edx+eax*8]
7fef045b10d: 666690 nop
7fef045b110: 48 dec eax
7fef045b111: 8d4940 lea ecx, [ecx+40h]
7fef045b114: 48 dec eax
7fef045b115: 8d5240 lea edx, [edx+40h]
7fef045b118: 4d dec ebp
7fef045b119: 8d40c0 lea eax, [eax-40h]
7fef045b11c: 0f db 0fh
7fef045b11d: 188280010000 sbb [edx+180], al
7fef045b123: f30f6f42c0 movdqu xmm0, [edx-40h]
7fef045b128: f30f6f4ad0 movdqu xmm1, [edx-30h]
7fef045b12d: 49 dec ecx
7fef045b12e: 83f840 cmp eax, 40h
7fef045b131: 660fe741c0 movntdq [ecx-40h], xmm0
7fef045b136: 660fe749d0 movntdq [ecx-30h], xmm1
7fef045b13b: f30f6f52e0 movdqu xmm2, [edx-20h]
7fef045b140: f30f6f5af0 movdqu xmm3, [edx-10h]
7fef045b145: 660fe751e0 movntdq [ecx-20h], xmm2
7fef045b14a: 660fe759f0 movntdq [ecx-10h], xmm3
7fef045b14f: 7dbf jge f045b110
7fef045b151: 4c dec esp
7fef045b152: 8d15b8010000 lea edx, [000001b8]
7fef045b158: 4d dec ebp
7fef045b159: 8bc8 mov ecx, eax
7fef045b15b: 49 dec ecx
7fef045b15c: c1e904 shr ecx, 04h
7fef045b15f: 49 dec ecx
7fef045b160: c1e104 shl ecx, 04h
7fef045b163: 49 dec ecx
7fef045b164: 03c9 add ecx, ecx
7fef045b166: 49 dec ecx
7fef045b167: 03d1 add edx, ecx
7fef045b169: 4d dec ebp
7fef045b16a: 2bc1 sub eax, ecx
7fef045b16c: 49 dec ecx
7fef045b16d: c1e904 shr ecx, 04h
7fef045b170: 0faef8 sfence
7fef045b173: 43 inc ebx
7fef045b174: ff24ca jmp dword ptr [edx+ecx*8]
7fef045b177: 66666690 nop
7fef045b17b: 66666690 nop
7fef045b17f: 90 nop
7fef045b180: 660f6f02 movdqa xmm0, [edx]
7fef045b184: 660f6f4a10 movdqa xmm1, [edx+10h]
7fef045b189: 660f7f01 movdqa [ecx], xmm0 <-- FAULT
7fef045b18d: 660f7f4910 movdqa [ecx+10h], xmm1
7fef045b192: 4d dec ebp
7fef045b193: 8d4080 lea eax, [eax-80h]
7fef045b196: 660f6f5220 movdqa xmm2, [edx+20h]
7fef045b19b: 660f6f5a30 movdqa xmm3, [edx+30h]
7fef045b1a0: 660f7f5120 movdqa [ecx+20h], xmm2
7fef045b1a5: 660f7f5930 movdqa [ecx+30h], xmm3
7fef045b1aa: 660f6f4240 movdqa xmm0, [edx+40h]
7fef045b1af: 660f6f4a50 movdqa xmm1, [edx+50h]
7fef045b1b4: 49 dec ecx
7fef045b1b5: 81f8a8000000 cmp eax, 000000a8
7fef045b1bb: 660f7f4140 movdqa [ecx+40h], xmm0
7fef045b1c0: 660f7f4950 movdqa [ecx+50h], xmm1
7fef045b1c5: 660f6f5260 movdqa xmm2, [edx+60h]
7fef045b1ca: 660f6f5a70 movdqa xmm3, [edx+70h]
7fef045b1cf: 48 dec eax
7fef045b1d0: 8d9280000000 lea edx, [edx+80]
7fef045b1d6: 660f7f5160 movdqa [ecx+60h], xmm2
7fef045b1db: 660f7f5970 movdqa [ecx+70h], xmm3
7fef045b1e0: 48 dec eax
7fef045b1e1: 8d8980000000 lea ecx, [ecx+80]
7fef045b1e7: 7d97 jge f045b180
7fef045b1e9: 4c dec esp
7fef045b1ea: 8d15d0ebffff lea edx, [ffffebd0]
7fef045b1f0: 49 dec ecx
7fef045b1f1: 03c8 add ecx, eax
7fef045b1f3: 49 dec ecx
7fef045b1f4: 03d0 add edx, eax
7fef045b1f6: 43 inc ebx
7fef045b1f7: ff24c2 jmp dword ptr [edx+eax*8]
7fef045b1fa: 666690 nop
7fef045b1fd: 666690 nop

Built on Aegis on Thu Dec 24 19:58:44 2009 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7600) []

RAX = 0
RBX = 5c0
RCX = 0
RDX = 37e4aaf0
RSI = 2d0
RDI = 5c0
RBP = 7fef05e4d00
R8 = 2d0
R9 = 200000
R10 = 0
R11 = 7fef0459dc0
R12 = f0
R13 = 0
R14 = 0
R15 = 37e4aaf0
RSP = 252ab408
RIP = 7fef045b189
EFLAGS = 00010246


Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'Veedub64'...

...writing address 0000000000000000.

Pointer dumps:

RDX 37e4aaf0: 10101010 10101010 10101010 10101010 10101011 10101010 10111110 10111110
RSP 252ab408: f01fce6d 000007fe 252ab440 00000000 365d1f30 00000000 365d1f30 00000000
252ab428: f020bdb9 000007fe 37e4aaf0 00000000 00000000 00000000 00000000 00000000
252ab448: 00000001 00000000 37e4aaf0 00000000 00000000 00000000 000005c0 00000000
252ab468: 05211fca 00000000 051337bc 00000000 f020b357 000007fe 365d1f30 00000000
RBP 7fef05e4d00: f01fbf50 000007fe f0201bd0 000007fe f0201b80 000007fe f0201aa0 000007fe
7fef05e4d20: f02019f0 000007fe f0305820 000007fe f0200d80 000007fe f0200c80 000007fe
7fef05e4d40: f01ff170 000007fe f01feda0 000007fe f01fe8b0 000007fe f01fe6f0 000007fe
7fef05e4d60: f01fe620 000007fe f01fe5d0 000007fe f01fdf20 000007fe f01fcec0 000007fe
R9 00200000: 00000000 00000000 001c2ad0 00000000 00200050 00000000 001fffd0 00000000
R11 7fef0459dc0: f045a4b4 000007fe f045a58f 000007fe f045a672 000007fe f045a757 000007fe
R15 37e4aaf0: 10101010 10101010 10101010 10101010 10101011 10101010 10111110 10111110

Thread call stack:
7fef045b189: avisynth!avs_at_exit [7fef01f0000+115680+155b09]
7fef01fce6d: avisynth!0000ce6d
7fef020b357: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4897]
05211b3d: fieldhint!00001b3d
76d7aac8: ntdll!TpCallbackMayRunLong [76d60000+1a3f0+6d8]
1b70dd7d: mt_masktools26!0005dd7d
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef01fcae8: avisynth!0000cae8
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
7fef01fcae8: avisynth!0000cae8
0516138f: telecidehints!0000138f
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef043c953: avisynth!avs_at_exit [7fef01f0000+115680+1372d3]
7fef01fcae8: avisynth!0000cae8
7fefcdb18aa: KERNELBASE!ResetEvent [7fefcdb0000+18a0+a]
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
1b6d1b22: mt_masktools26!00021b22
7fefcdb10ac: KERNELBASE!WaitForSingleObjectEx [7fefcdb0000+1010+9c]
7fef025a559: avisynth!DllGetClassObject [7fef01f0000+20ec0+49699]
7fef01fce6d: avisynth!0000ce6d
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fefcdb18aa: KERNELBASE!ResetEvent [7fefcdb0000+18a0+a]
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
05193f70: decomb!AvisynthPluginInit2 [05190000+1000+2f70]
7fef01fc9cc: avisynth!0000c9cc
7fef01fcae8: avisynth!0000cae8
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef01fcae8: avisynth!0000cae8
7fefcdb134c: KERNELBASE!SetEvent [7fefcdb0000+1340+c]
7fef020b6e7: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4c27]
7fef025a559: avisynth!DllGetClassObject [7fef01f0000+20ec0+49699]
7fef01fcae8: avisynth!0000cae8
7fefcdb134c: KERNELBASE!SetEvent [7fefcdb0000+1340+c]
7fef020b6e7: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4c27]
051918e2: decomb!AvisynthPluginInit2 [05190000+1000+8e2]
7fef01fc9cc: avisynth!0000c9cc
76db1c58: ntdll!RtlAllocateHeap [76d60000+51b70+e8]
7fef020b6e7: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4c27]
051917b8: decomb!AvisynthPluginInit2 [05190000+1000+7b8]
7fef043fa1f: avisynth!avs_at_exit [7fef01f0000+115680+13a39f]
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef043c953: avisynth!avs_at_exit [7fef01f0000+115680+1372d3]
7fef01fcae8: avisynth!0000cae8
7fefcdb18aa: KERNELBASE!ResetEvent [7fefcdb0000+18a0+a]
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
7fef01f7e22: avisynth!00007e22
7fef043fa1f: avisynth!avs_at_exit [7fef01f0000+115680+13a39f]
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef043c953: avisynth!avs_at_exit [7fef01f0000+115680+1372d3]
7fef01fcae8: avisynth!0000cae8
7fefcdb18aa: KERNELBASE!ResetEvent [7fefcdb0000+18a0+a]
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
7fef025a559: avisynth!DllGetClassObject [7fef01f0000+20ec0+49699]
7fef043fa1f: avisynth!avs_at_exit [7fef01f0000+115680+13a39f]
7fef0209ad4: avisynth!avs_release_video_frame [7fef01f0000+16ac0+3014]
7fef043c953: avisynth!avs_at_exit [7fef01f0000+115680+1372d3]
7fef01fcae8: avisynth!0000cae8
7fefcdb18aa: KERNELBASE!ResetEvent [7fefcdb0000+18a0+a]
7fef020b655: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4b95]
7fef01fcae8: avisynth!0000cae8
7fef0259006: avisynth!DllGetClassObject [7fef01f0000+20ec0+48146]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef020b6e7: avisynth!avs_release_video_frame [7fef01f0000+16ac0+4c27]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]
7fef0258ff0: avisynth!DllGetClassObject [7fef01f0000+20ec0+48130]

-- End of report

[turbojet]

osgZach: You didn't make a mistake on using the installer, I did. I didn't account for more spaces in the path here's a fixed install script and an uninstall script (http://www.mediafire.com/?zthdjndw5wo). All you should have to do is run the fixed installer, move any plugins from C:\Program 2.5\plugins64 to the avisynth install directory and delete C:\Program 2.5.

JoshyD: Can you remove the old broken install script and include the fixed install script and uninstaller in the next package? Also Do you have any news on an 'amd fix'? If it helps any I can test a 2.6 cvs dll if you can compile it, currently I don't have the time or the patience to setup a compiling environment again.

I was the one that asked for Decomb v4, for more reliable field matching, deinterlacing results then v5 but TIVTC would definitely trump both of them if time permits it.

[osgZach]

Thanks for that fix. I'll download it in a bit and give it a whirl.

Also.. if anyone could give advice, or point to a thread where it is discussed.. I'm interested in either completely moving to a 64-bit encoding/playback path, or just looking for some general guidelines on how to set it up easily so that my 32/64 bit stuff can peacefully co-exist and I can keep track in a way so that I know what's what, when working in scripts / other programs. As of right now I'm new to this so obviously I've had to do some juggling around of files, etc with scripts errors for this or that reason. Mainly I am worried that I am gonna make my system barf, and kill my brain trying to keep track of both x32/x64. I have no experience manually installing or things like ffdshow, Haali, etc or fussing with registry entries, etc.. I use CCCP and I don't think they have a pure 64-bit version?



Of course going pure 64-bit I guess depends on whether that Vdub crash was just a random oddity, has to do with Vdub itself, or the x64 Avisynth /or one of the x64 filters..

Though considering my (lack of) knowledge, perhaps it would be safer/better for me to maybe try getting Avisynth x32 MT setup instead? that seemed to be where the bulk of my performance boost came from. Is it a relatively easy process like installing x64? (a couple files copied, etc). It was great fun playing around with for sure.. But I gotta have a stable setup so I can leave the PC on and encoding, and not come back hours later to some random crash that ironically happened just after I left the room :p

[turbojet]

As for setting up I'd suggest this haali media splitter (http://www.mediafire.com/?yy5nimzdwed) (only one I've found that has an x64 splitter and splits VC-1) and ffdshow64 (http://sourceforge.net/projects/ffdshow-tryout/files/SVN%20builds%20by%20clsid/64-bit%20builds/ffdshow_rev3305_20100304_clsid_x64.exe/download). This should decode just about everything as long as a decoder is set in ffdshow decoder configuration. If you use windows 7 Preferred filter tweaker (http://www.codecguide.com/windows7_preferred_filter_tweaker.htm) will help set the defualt decoder.

With avisynth store 32 bit plugins in plugins, 64 bit in plugins64. The same avs script can be used for both and what determines if it's 32 or 64 bit is what application opens the avs file.

[hajj_3]

this 64bit avisynth is looking promising :) Alot of progress has been made in such a short space of time. Anyone know if LeakKernelDeint is open source and if it has a 64bit version compiled?

[Guest]

this 64bit avisynth is looking promising :) Alot of progress has been made in such a short space of time. Anyone know if LeakKernelDeint is open source and if it has a 64bit version compiled? Source is here (scroll to bottom):

http://neuron2.net/kerneldeint/kerneldeint.html

[osgZach]

As for setting up I'd suggest this haali media splitter (http://www.mediafire.com/?yy5nimzdwed) (only one I've found that has an x64 splitter and splits VC-1) and ffdshow64 (http://sourceforge.net/projects/ffdshow-tryout/files/SVN%20builds%20by%20clsid/64-bit%20builds/ffdshow_rev3305_20100304_clsid_x64.exe/download). This should decode just about everything as long as a decoder is set in ffdshow decoder configuration. If you use windows 7 Preferred filter tweaker (http://www.codecguide.com/windows7_preferred_filter_tweaker.htm) will help set the defualt decoder.

With avisynth store 32 bit plugins in plugins, 64 bit in plugins64. The same avs script can be used for both and what determines if it's 32 or 64 bit is what application opens the avs file.

Thanks.. I'm currently trying to do some further testing to see if anything on my end is causing this Access Violation.

So far its codec independent that I can tell. Lagarith does it, no matter the settings. I also came across UT Lossless Codec (http://forum.doom9.org/showthread.php?p=1382276) which is YV12 and multithreaded, which makes it a great replacement for HuffYUV until I can get FFDshowx64 installed (hopefully w/out problems) and see if they have an x64 HuffYV12 or not.

But it still does it. It seems like its either Avisynth causing the crash, or one of the filters like MT (not sure if its "MT" or masktools) or something.

I am currently running the script without SetModeMT(2,0) to see if it throws another Access Violation or not.

[osgZach]

It's definitely a crash caused by SetMTMode(2,0)

Encode without this line completed successfully..

Are there other MT modes I can try?

[levi]

It's definitely a crash caused by SetMTMode(2,0)

Encode without this line completed successfully..

Are there other MT modes I can try?

The modes are described @ http://avisynth.org/mediawiki/MT

usually i do
SetMTMode(3,0)
...source(...)
SetMTMode(2,0)
...


JoshyD

not sure if you saw it, but Set did make 2.6 avisynth build as well - http://forum.doom9.org/showthread.php?t=148782

Requests: (since you are so kind to accept them) :)
I'd like to cast another vote for 64 bit TIVTC & leakkerneldeint (in that order)

Also, someone mentioned repal but linked to the older version, the newer version (outputs 25 fps) is found at http://forum.doom9.org/showthread.php?p=1092552#post1092552 however srestore (http://forum.doom9.org/showthread.php?t=95924) is much more likely to give positive results across a variety of sources so i would not put repal at the top of a priority list

I want to test this with MCTemporalDenoise (http://forum.doom9.org/showthread.php?t=139766) which is a slow script I like.

Thanks for your efforts! This is a very exciting development. :)

[JoshyD]

@osgZach
What script/filters were you trying to run when you experienced the crash? That looks like a memory copy routine followed by some function, and that function looks vaguely familiar. There are MT modes ranging from 1 to 5, depending on how the filter wants to access data, the modes offer varying amounts of "protection" from events occurring asynchronously. Masktools *should* be thread safe, but that I can dig further. From the callstack, it looks like something wasn't cleaning up after itself correctly, I bet it's an easy fix.

@turbojet
I don't have access to an AMD machine to test why this is failing. Did you try the really old SSE2 only build from some time ago I linked in a previous post? If that's still failing, I'm going to have to find someone with an AMD machine to let me debug with. I'll change the installer in the next update. I've found a few other bugs and remnants of code that shouldn't exist in the 64bit environment. These changes aren't resize related though, this one's really bothering me. I also have a new Masktools that needs to go up after some testing, and ColorMatrix to post as well. I can get 2.6's CVS built and up sometime tomorrow, and then it's the weekend :)

[osgZach]

Here is the AVS, with the SetMTMode line that makes it crash..
I left out the truck-load of Trim statements at the end, for obvious reasons, but the rest is intact.

Also.. SetMTMode other than 1 or 2, crashes immediately (and it points to masktools 2.6). 1 loads with corruption but obviously crashes when you try to do anything.


SetMTMode(2,0)
LoadPlugin("C:\yatta\plugins64\decomb.dll")
LoadPlugin("C:\yatta\plugins64\dgdecode.dll")
LoadPlugin("C:\yatta\plugins64\telecidehints.dll")
LoadPlugin("C:\yatta\plugins64\fieldhint.dll")


function Preset0(clip c) {
#Name: Default
c
return last
}

DGDecode_Mpeg2Source("L:\Ep 01\VTS_01_1.d2v")



FieldHint(ovr="L:\Ep 01\VTS_01_1.d2v.fh.txt")
TempGaussMC_beta2().SelectEven()

PresetClip0=Preset0()

PresetClip0.Trim(0,41023)



DClip = Decimate(cycle=5,quality=3,ovr="L:\Ep 01\VTS_01_1.d2v.dec.txt").assumefps(last.framerate)


Several variations on the line calling TGMC (I Believe with TelecideHints also) will still cause a crash as well. Although I forgot which specific variation it originally crashed under (was pretty tired yesterday when fiddling with this stuff).

It's kind of a bummer really. I don't really get much of a speed improvement at all that I can tell, with x64. Most of it seems to come from using SetMTMode. But I'm currently doing more testing as my understanding allows.

I did visit the avisynth page, and one thing that I really couldn't "follow" was the MTi() stuff listed before SetMTMode.. I've never seen anyone actually use that stuff in a script either, always just the SetMTMode.. is it of any use in this situation?

8:50am EST:
I'm currently trying without SetMTMode, using this line instead

MT("TempGaussMC_beta2().SelectEven()",threads=2)


If it works, it will have cut work time, solidly in half. (nearly 5 hours, down to around 2 and a half per file). Hopefully it won't crash or anything, but will report back around 11am (or before) or so if it has.

[aegisofrime]

I finally got around to trying your 64-bit plugins JoshyD. I would have done it far sooner, but was worried about breaking my 32-bit Avisynth.

On a 624x352 test clip with the following script


DirectShowSource("D:\Test\Test.avi")
tempgaussmc_beta2(1,1,0,0,0,0,edimode="--",SVthin=0.0,pelsearch=1)

64-bit TGMC yield 14.29 FPS. The 32-bit version yielded 12.99.

For 64-bit I used x264_64 CLI. For 32-bit I used Lord Mulder's x264 launcher, so that I could use 64-bit x264 and thus reduce the variables to purely 64-bit vs 32-bit TGMC.

The test platform is a Core 2 Duo P8700 notebook with 4GB DDR3 RAM running Windows 7 (64-bit of course!)

It might not seem much, but I believe on a more powerful rig and over the course of a full length DVD movie the difference can be quite significant. I will get around to testing it on my Phenom II 955 rig and see if Intel's compiler really do cripple AMD performance.

I did run into a problem though. I tried to run FFT3DGPU but ran into an error with a missing file d3dx9_42.dll. Downloading that file and placing it in both my SysWOW64 and System32 folders merely gave me an error loading FFT3DGPU.dll.

In any case, thanks for the great work JoshyD! It's amazing what you have done on your own.


8:50am EST:
I'm currently trying without SetMTMode, using this line instead


If it works, it will have cut work time, solidly in half. (nearly 5 hours, down to around 2 and a half per file). Hopefully it won't crash or anything, but will report back around 11am (or before) or so if it has.

I tried that before, at least on the 32-bit version. TGMC does not work with the MT() function. It messes up the picture. If it works for you I would like to know though!

[Stephen R. Savage]

@aegisofrime: You need the 64-bit version of the d3dx9_42.dll and the website you downloaded it from probably only offered the 32-bit version. Have you tried using the Microsoft DirectX updater/installer?

Also, note that you introduce a bias by using x264 to compare speeds because the 32/64 piping introduces overhead, which may in fact be equal to the 1.3 fps advantage you see (though probably not).

[aegisofrime]

@aegisofrime: You need the 64-bit version of the d3dx9_42.dll and the website you downloaded it from probably only offered the 32-bit version. Have you tried using the Microsoft DirectX updater/installer?

Also, note that you introduce a bias by using x264 to compare speeds because the 32/64 piping introduces overhead, which may in fact be equal to the 1.3 fps advantage you see (though probably not).

Thanks for the reply. I have not needed to use the DirectX updater for quite a long while, but I will give it a try soon.

With regards to my testing methology, that's possible, although when MeGUI introduced 64-bit x264 support most agreed that the speedups from 64-bit x264 outweighs any latency issues with piping.

I think the next best thing is to use Veedub64 to do a Lagarith lossless encode? Is that what "save as AVI" does? Because it's the first time I'm using that software...

[Stephen R. Savage]

Yes, "Save as AVI" in VirtualDub compresses the video using VfW. The problem here is that now you have to factor in the cost of 32-bit vs 64-bit encoders. That is why I recommend avs2avi.exe with null output for comparing speed, because it eliminates outside variables.

avs2avi test.avs NUL -c null -o n

[aegisofrime]

Ah, the line you provided gave me the error "There is no codec with FourCC "null"!"

Pardon the noobness, again it's my first time using Avs2Avi too!

[osgZach]

aegisofrime,

I don't know what to tell you but from what I can tell my encode has come out perfectly.

2 hours 9 minutes and 30 seconds (give or take). Averaging about 4.5fps across 33,328 frames. Using MT() versus SetMTMode(). The same encode under 32-bit with the same filter and no MT would take around 4h:45m.

This makes me want to start priciing Core 2 quads at Newegg, to replace my Core 2 Duo :D But I must persevere! Fermi will be out soon and I'm on an aging 8800GTS(g92) 512MB. Although after selling my old 19" Samsung and maybe the C2D I could make up most of the cost anyway.... hmmm.. decisions ;)

Were you doing any manipulation in your script? I know I read that MT will totally screw up if you try to resize Width or Height or something like that..

btw Joshy.. Spline64Resize does not work ?? I'll test it again but it was Avisynth throwing me errors last time I tried it.

Also for anyone interested.. UT Codec is pretty awesome from what I can tell. File comes out much smaller than HuffYUV/12 probably on par with what Lagarith could do, or a bit smaller?
Its pretty nice either way.


General
Complete name : G:\test1.avi
Format : AVI
Format/Info : Audio Video Interleave
Format profile : OpenDML
File size : 3.88 GiB
Duration : 18mn 32s
Overall bit rate : 29.9 Mbps
Writing library : VirtualDub build 32706/release

Video
Format : ULY0
Codec ID : ULY0
Duration : 18mn 32s
Bit rate : 29.9 Mbps
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 1.500
Frame rate : 29.970 fps
Standard : NTSC
Bits/(Pixel*Frame) : 2.890
Stream size : 3.88 GiB (100%)

[aegisofrime]

It works? I will have to give it a go myself then. Although I seem to recall Didee saying it won't work either. Perhaps it's because I have SetMTMode(2,4) at the top of my script and a Distributor behind. That gives me around 13 FPS on a Lossless Rendering run with TGMC(2,1,1,"NNEDI2" on my Phenom II 955.

Them Core 2 Quads ain't bad but I would really consider a Core i5-750 at least... Although that will require a motherboard change... And I'm waiting for Fermi to replace my 4870 as well!

[osgZach]

Yeah there is a really nice Lynnfield i5 up there w/8MB cache and virtualization support (tho no hyperthreading, but still 4 cores).

The SetMTMode gave me problems so maybe that is why it messed up on yours too.. hmm. Although we have different issues so who knows.
Now we need some 64-bit filters for Dot Crawl, Rainbows, etc.. I don't know how to do any of that with the filters I've collected so far.. Which by the way really need to be distributed with their Readme's..

I also can't get FFT3D/3DGPU to work.. comaplains about fftw3.dll, but I dunno. Its in the plugins64, system32, syswow64.. should all be the same version too. Complains it can't load it though.. lol. Not that I use it that much.
Think I can make due without most of the other filters for now tho.. Death Note's DVD have horrible color banding or something that I just can't fix, even with the proper filters. They go away if you use Gradfun2db to add grain, but then when you try to denoise or sharpen up in any way, they come right back.. not that they dissapeared much to begin with.. LOL Sorry for off topic.

Gonna go track down some AA bits to try w/EEDI2 there were some good fast ones in a thread somewhere around here (maybe one of mine).

Keep up the good work JoshyD.

p.s what I meant by a one-click installer was starting blank with no Avisynth installed. But having a installer package that will explicitly install an x64 only version.. I assume its just modifying the current installer with the changed files or something?

[osgZach]

Hmm ok Aegis, maybe you were right.. I just noticed, on frames that appear to be scene changes and fades (or fades on scene changes or something like that, but definitely scene changes) there is a very thin black line going DEAD CENTER through the image. (across the width, not the height i.e divides into a "top half" and "bottom half")
I am guessing this is an MT and/or Bobbing artifact.. An Overlap problem? Can that be tweaked ? It doesn't happen on every single frame in sequence, it might disappear and reappear, but once the scene is established it appears to go away..

Would MTi be better in this case? Advice appreciated. Frankly I never noticed it during playback, even when looking for it. So it may not be a huge issue, but would be nice to fix.

Fixed: (hopefully)
Hmm.. overlap 4 seems to give the best results. Aegis, if your problem was similar try tacking that MT("filter()",threads=int,overlap=int)

[JoshyD]

@osgZach
I seem to remember DGDecode not liking SetMTMode for some reason. If you want to try the same script, do something like this

SetMtMode(5,0)
DG_decode(<path>\source)
SetMtMode(2)


Setting an MT mode before calling a source multithreads accesses to the source itself. In this case, setting mode 5 before the source will make it behave like a single threaded source, while all other filters after SetMTMode(2) will behave nicely.

The version of FFT3DGPU linked on the main post shouldn't need that dll, but the regular FFT3DFilter should have been packaged with the fast fourier transform library needed to run correctly. Turns out I packed the wrong DLL with the filter, my mistake. The package is updated, if you'd grab it and give it a a whirl, I'd much appreciate it.

The FFT3DGPU filter DOES need a different *.hlsl file to be in the same directory as the plugin. I re-wrote the shaders used to be compliant with newer DX9 syntax rules. The results are the same though, but FFT3DFilter produces overall higher quality results in my opinion. Special note for FFT3DGPU, if you want to use SetMTMode around it, put the script in SetMTMode(5) before using the filter, and change back when you're done calling it.

As for Spline64Resize, mine works correctly for the test clips I've thrown through it. All of the resize functions eventually hit the same code, perhaps there's some corner case that you are running into? What's the size (widthxheight) and color space of the clip that's causing the resize to complain? I guess, all the video information on the clip as well as what you were trying to size it to would be helpful.

Finally, with the masktools 2.6 crashing, I'm pretty sure I can get that fixed. There's a new rough build of it as of now. See if that helps, otherwise, I may be going back in and re-writing the belly of the thing. Masktools uses an assembly macro to process all function calls, the problem being that 64bit C++ calls are all fast calls, and the function calling/returning code in masktools assumes a __cdecl call, I think. It's a pretty integral part of the whole thing, so I went about modifying it as gingerly as I could.

Masktools and MVTools are two plugins I'd really like to go back and optimize further, but I want to get a working TIVTC out, and before that, I'd really like to figure out why turbojet's Athlon II bites the dust when resizing. I'd also like to re-write EEDI2 for some extra speed all around, but these are more long term goals.

@aegisofrime
You've got this running on a Phenom II?? turbojet's been having troubles with his Athlon II. If I can get confirmation that this release doesn't break AMD processors, I'd be a very happy dude.

I think the syntax you want to use for benchmarking with avs2avi is:avs2avi64.exe <path:\script.avs> -o n

Choose no compression from the little dialog that pops up and wait. This will just run your script/video through avisynth, without writing the results to disk. Compare final FPS reports, and judge speed improvements accordingly.

Now to clear a few things up on multithreading:

The differences between MT("function") and SetMTMode aren't very well documented. Quick explanation, SetMTMode creates multiple instances of the filters used and allows more than one frame to be worked on at once. The drawback here is, that in some cases two functions will be trying to modify the same "things", giving you that nasty access violation. The different modes of SetMTMode (1-5) are designed to safegaurd against this. Mode 5 essentially puts whatever filter you've called into single threaded mode. Mode 1 creates a free for all when it comes to memory access/modification, which I don't think is really ever going to work well. Special note on use of SetMTMode, the number of threads can only be set during the first call to the function. Any use of SetMTMode(x, threads) after the initial use will ignore the "threads" parameter.

MT(clip,"function",threads,overlap) splits your source into pieces based upon how many threads you specify. For instance, clip.MT("FFT3DFilter()", threads=2), divides the source into top and bottom halves, processes them each with a different thread, and laces the output back together. The split (you can choose whether to do it horizontally or vertically) limits your options for resizing, because it doesn't know how to re-stitch together frames that started at one size, and then ended up completely differently sized at the end.

The other problem with using MT("function") is that the two halves may present different complexities, and therefore have different processing times. Also, if using a filter that does overall motion compensation or something of that sort, you only get half the information. The overlap parameter helps to alleviate this problem by making sure that both portions of your clip share some boundary data. If anyone remembers 3D game SLI (literal scan line interleaving 3DFX Voodoo2 style) vs load balanced rendering based on scene complexity, The same ideas can be applied to SetMTMode and MT("function"). There's no universal solution, that's why nVidia/ATi have custom profiles for different games.

A note on MTi, this operates essentially the same as MT, instead giving the filter the hint that you are feeding it an interlaced source. The internal functionality of the filter uses this hint to produce "more correct" results. (I think it may just separate fields before it goes calling other functions, call the functions with only the even frames and only the odd frames, finally weaving the results together)

Total tangent: As for current processors, Core i5-750 all the way. Hyperthreading is overrated, it's there to ensure poorly written code can keep all the execution units available busy. Write good code, and this shouldn't happen. The i5-750 overclocks like nobody's business. I've got mine, using the diminutive stock cooler and at stock voltage, running just north of 3.7GHz, stock speed is 2.66GHz. I think it's an upgrade that will last a while. There's some nice new technologies on it, not to mention that it was designed with four cores in mind, giving all four cores a pooled (and large) level 3 cache on die. The Core2Quads are two Core2's on the same die, so each pair of cores share a cache, but need to talk via an external bus to figure out which core has what in their cache, and synchronize accordingly.

I'm a computer architecture nut. The assembly functions that crop up in the 64bit version of Avisynth were written after a fair deal of reading of the Intel processor white papers. It's an attempt to optimize the code based on it's actual execution, taking into account which x86 micro-ops fuse, how a branch predictor will handle looping functions, memory accesses, register renaming, reorder buffers, pipeline stalls and flushes, blah blah blah.

[osgZach]

Interesting. I know earlier today I -did- try playing around with a SetMTMode initially 5, and then 2 around the filter but that still seemed to mess up or not work, etc.. I will go back and revisit the idea again and make sure I did everything properly.. Also is it necesarry to have a SetMTMode at the very top of the script, always? Or can I just call it AFTER DGDecode ? In either case I'll see if I can think of something.

MTi is an interest solution, but I don't have a true interlaced source.. not in the sense that every frame is interlaced, if I understand correctly. Most of its just combs from Telecine.. would that matter?

I will try and get back to you on Spline64Resize, as I know I could not get it to work that I can remember.. Spline36 will though.. Although it does funny stuff if you tag it onto the end of an EEDI2 anti-aliasing line (turnleft, blah blah, turn right, etc).

This baby IS very tempting I agree.. (http://www.newegg.com/Product/Product.aspx?Item=N82E16819115215&cm_re=Core_i5-_-19-115-215-_-Product) I also like that it supports virtualization technology too as its fun to play around with.. I think I saw a decent bundle for <$400 but that would also mean buying new DDR3... meh.. Fermi first, I'm pretty happy @ 3.2Ghz just need a better GPU for the time being :D

Edit:

I've done this is my script thus far but the performance seem lacking... although it takes a while for it to stabilize and pickup (its only been 20m).. But I'm only average around 2.5 to 3.5fps with ETA @ 2:42m so far. Which is a 30m increase..


SetMTMode(5,0)
DGDecode_Mpeg2Source("L:\Ep 01\VTS_01_1.d2v")
SetMTMode(2)


FieldHint(ovr="L:\Ep 01\VTS_01_1.d2v.fh.txt")

#MT("TempGaussMC_beta2().SelectEven()",threads=2,overlap=4)

TempGaussMC_beta2().SelectEven()

PresetClip0=Preset0()

PresetClip0.Trim(0,41023)

DClip = Decimate(cycle=5,quality=3,ovr="L:\Ep 01\VTS_01_1.d2v.dec.txt").assumefps(last.framerate)

Above the MT are LoadPlugin calls, an of course I left out the billion or so Trim/ Dclip.Trim statements

Setting the MT at the top of the file did no good, so I'm not sure if that is necesarry (I thought I read that somewhere). It gives only single thread performance with a few spikes up to 80 - 100% usage here and there. Whereas the script I have going now never drops out of the 90's (and I know its not a good guage, but high usage at least tells you its working, when its above 50).

Hopefully I'm doing it properly now, though.. and hopefully it won't error out on me since DGDecode is no longer being threaded erroneously.

Still, MT("filter",threads,overlap) seems to be faster.. I know the overlap isn't perfectly curing it, but you have to blow it up to %200 to even see a small trace of the merge halves and its not even a uniform line.. I could potentially live with that, and hell it might even be my eyes deceiving me and just be other noise.

I don't think its the threading overhead from UT Codec either, as I've been using it in all my tests since the crashing stopped.

Hoping it will ease up and pan out to no more than 2h:30m when all is said and done though.. but it's inching its up way to the 3hr mark for ETA :(

[osgZach]

Access Violation... again, heh..

Is this starting to look like a Mask/MV Tools issue ?

[kemuri-_9]

What does your avs build look for as the entry point for C Plugins?
A) avs x86 looks for avisynth_c_plugin_init[@4]
B) squid_80's avs x64 build technically looks for avisynth_c_plugin_init_s[@4] (minus the part where it doesn't actually have LoadCPlugin enabled)

was wondering if you altered this to be like squid_80's build or if you left it as is and are looking for the same thing the x86 version looks for.....

relevant as I'll need to know what entry point i need to define for my avs64 branch of ffms2. (that uses the avs C plugin interface)

[osgZach]

Updated FFT3DFilter now works properly. :)

Also, I tried Setting the mode back to 5 again, after I set it to 2 for TGMC, thinking maybe it was something past that causing it to goof up.

But if I do that, it never goes multi-threaded.

Can you not have more than two SetMtMode declarations in a script?

[kemuri-_9]

you need the final MT mode to be 0 < mode < 5 for avisynth to call distributor() automatically.
otherwise you could try adding Distributor() to the end of your script yourself if you want it threaded.

[osgZach]

0 < mode < 5


Sorry but I don't quite understand how that should read.

[JoshyD]

@kemuri-_9

No alteration to the loading of c plugins on my part, SEt hacked c-plugins to identify themselves as cache, I think. Take a look at the source to see if it gels with what you were expecting. I don't know if anything is different off-hand, nor even if it loads C-plugins correctly, as I haven't had any to try. I'm not sure if this makes a difference at all, but under x64 all function calls are fastcalls. I'm betting you already knew that . . .

[aegisofrime]

@JoshyD : I'm sorry if I wasn't clear. I tried the 64-bit stuff on my C2D notebook as I was afraid to break any stuff on my Phenom II right which is my main encoding rig. Now that I have gained confidence with it I will be trying it on that rig soon. It's currently encoding something now... So I will report back when I have tried.

With regards to avs2avi64, I assume you are talking about plain avs2avi instead? Because Googling for avs2avi64 gives your post as the only result. :D In any case, I tried your syntax, and as there was no option for "No Compression" I chose full frames (uncompressed) which gave me a Valid Compressor not chosen error. Perhaps my version of avs2avi is outdated, I'm going to poke around to see if I can find a newer one.

EDIT: I need to learn to read instructions more carefully. I missed the link which you provided in your first post!

EDIT2: I HAVE RESULTS!

32-bit : 28.02 fps
64-bit : 34.18 fps

I'm quite pleased!

Yet another edit (because I don't want to double post): Has anyone tried the 64-bit version of DGDecode at squid80's site? It gives me an unable to load DGDecode error, even though I have placed the required libmmd.dll file in the system32 folder :(

[kemuri-_9]

Sorry but I don't quite understand how that should read.

modes that are >0 and <5 (1 through 4 inclusive)

No alteration to the loading of c plugins on my part, SEt hacked c-plugins to identify themselves as cache, I think. Take a look at the source to see if it gels with what you were expecting. I don't know if anything is different off-hand, nor even if it loads C-plugins correctly, as I haven't had any to try. I'm not sure if this makes a difference at all, but under x64 all function calls are fastcalls. I'm betting you already knew that . . .

trying to load my ffms2 x64 C plugin causes avisynth to crash at startup.

i wrote a C++ plugin to add LoadCPlugin/Load_Stdcall_Plugin functionality (to use originally with Squid_80's build since he didn't include these functions in his though the entire C plugin framework was there and functioning).
and loading my ffms2 C plugin via this plugin has no issues and it works as intended.

you seem to have simply ported over the stack corruption checking that's supposed to check for the plugin being stdcall (and following that convention)
which you had already pointed out that win x64 does not follow this convention so what's going on here?

[turbojet]

JoshyD: The SSE2 build works. I don't think SSE2 is what AviSynth 2.60 is using during resize judging by these results

2.58-x86: 51.51
2.60a2-x86: 57.77
+12%

3-1-10-x64: 49.25
SSE2-x64: 51.07
+3%

If you plan on suggesting the SSE2 build for AMD CPU's for now you might want to link libiomp5md.dll if you can, it's not an easy dll to find.

[Gavino]

The differences between MT("function") and SetMTMode aren't very well documented.
See http://avisynth.org/mediawiki/MT_support_page, especially section 2.

[osgZach]

Well I've done some testing.. I think SetMTMode 3 stopped the crashing (and it was occuring not because of DG_decode, but because of either Decimate, or Trim).

However any suitable combination of MT Modes aside from a flat out Mode 2 blanketing the entire script - has worse performance than using what I did before with
MT("function",threads,overlap), we're talking almost no improvement over not using MT at all, or possibly worse.

So I guess I'm stuck with that.. in all honest I think its pretty darn good.. And not anything you would notice at sizes 100% or smaller if in fact there are any leftover artifacts from the split-join. It's worth it from my perspective I guess, as it seems to run without problems and cuts the time in half at least.

[JoshyD]

JoshyD: The SSE2 build works. I don't think SSE2 is what AviSynth 2.60 is using during resize judging by these results

2.58-x86: 51.51
2.60a2-x86: 57.77
+12%

3-1-10-x64: 49.25
SSE2-x64: 51.07
+3%

If you plan on suggesting the SSE2 build for AMD CPU's for now you might want to link libiomp5md.dll if you can, it's not an easy dll to find.

It isn't . . . it uses the same opcodes my resizers use, which is why I'm at a loss as to why my resizers cause your computer to error.

Avisynth 2.6 uses SSE3 as long as the FIR filter size is below 8, which is a pretty decent cutoff. Resizers that need larger filter sizes are normally pretty rare. TGMC beta 2 goes up to size 9 for my test vectors.

Also, my compiler settings shouldn't allow your Athlon down any Intel code paths. The intel specific code paths, from what I can gather by reading icc vs gcc posts, are only executed when an intel family processor is detected. Any of Intel's "special" code should look at your AMD processor, read the vendor ID, and send it down a generic code path.

Also, aegisofrime seems to indicate that he has it running on his Phenom II x4 in the post above. kemuri-_9's system specs indicate he's running a Phenom II x4 as well, and he hasn't voiced any resize specific problems as of yet. Your Athlon II is a stripped down phenom II core, I believe. I'm a bit baffled.

I usually link Open-MP statically, that SSE2 only build is really old and I probably omitted the compiler flag for it. It has to be manually entered, because Open-MP, when used by multiple plugins, will error if they're all statically linked. For this reason, I haven't built any of the plugins with Open-MP directives. EEDI2 in particular enjoys a massive speed gain if you let it run with multiple threads.

you seem to have simply ported over the stack corruption checking that's supposed to check for the plugin being stdcall (and following that convention)
which you had already pointed out that win x64 does not follow this convention so what's going on here?

You are correct, I didn't touch any of these routines, they'll definitely need to be changed. I hadn't even thought about loading C plugins with this dll until you expressed interest in getting a 64bit port of FFMS2 working. I was updating on an "as needed" basis, it's just a lot of code to sift through, I can't get it all in one shot. I grabbed the FFMS2 source, are you building it with MinGW? There's a MSVS project in the svn checkout, but even with C99 support, I'm missing some headers, and wanted to be consistent with your build environment, for testing/debugging and such. A compiled x64 dll would let me run through my source to tie any loose ends up, if you could either a) just link an x64 dll or b) fill me in on how you're compiling, I'd greatly appreciate it, and can get the avisynth core changed ASAP.

[kemuri-_9]

Also, aegisofrime seems to indicate that he has it running on his Phenom II x4 in the post above. kemuri-_9's system specs indicate he's running a Phenom II x4 as well, and he hasn't voiced any resize specific problems as of yet. Your Athlon II is a stripped down phenom II core, I believe. I'm a bit baffled.


I've been mostly sitting on the sidelines on this and not actively testing so don't pull me into any arguments as proof of something!
(I only got the binary and started working with it for trying the ffms2 plugin just yesterday!)
I have 3 pcs consisting of PhenomII x4, Phenom x4, and athlon64 x2 all running on x64 versions of windows, so i can do testing from the AMD side of things if the need arises....
(as i usually do this for x264 as the other x264 devs mostly use Intel and/or linux)

You are correct, I didn't touch any of these routines, they'll definitely need to be changed. I hadn't even thought about loading C plugins with this dll until you expressed interest in getting a 64bit port of FFMS2 working. I was updating on an "as needed" basis, it's just a lot of code to sift through, I can't get it all in one shot. I grabbed the FFMS2 source, are you building it with MinGW? There's a MSVS project in the svn checkout, but even with C99 support, I'm missing some headers, and wanted to be consistent with your build environment, for testing/debugging and such. A compiled x64 dll would let me run through my source to tie any loose ends up, if you could either a) just link an x64 dll or b) fill me in on how you're compiling, I'd greatly appreciate it, and can get the avisynth core changed ASAP.

yes, I'm building with MinGW completely as per the reasoning of http://doom10.org/index.php?topic=25.msg1730#msg1730
ffms2.dll: x64 testing binary (http://kemuri9.net/dev/avs/ffms2/x64/ffms2.dll)
aforementioned LoadCPlugin plugin: x64 binary (http://kemuri9.net/dev/avs/LoadCPlugin/LoadCPlugin.dll) src (http://kemuri9.net/dev/avs/LoadCPlugin/LoadCPlugin.zip)

[JoshyD]

@kemuri-_9
Being the king of AMD PC's that you are, would you mind pulling a quick vertical resize test on any source to see if it's broken across the board for AMD users? If you could pull the latest binary (I uploaded a new one today), it'd be helpful to see if something funny is happening on the AMD side of things. You'll need it to properly test ffms2 anyway :).

Giving a quick look at your loadCplugin vs the current one that checks stack corruption, I decided to just drop your LoadCPlugin function in as a replacement. As a result, I've got your ffms2.dll loading and the few tests I've run (various sources with some post processing effects, etc) have it running, with some oddities. Running a debug build of avisynth through through MSVS's debugger points to some code in ffms2.dll that is making illegal memory accesses. However, frames come through and appear correctly when running a release build of avisynth. There are no memory access violations thrown.

The oddity is that the 1st frame (frame 0) comes through as garbage. Moving around the source a bit, and then coming back to frame 0 has the frame rendering correctly. I figured I'd let you look it over, it may be my avisynth not initializing your plugin correctly. As you've probably gathered, I'm not too familiar with the plugin loading code of avisynth. My main focus has been on optimizing the calculation and memory heavy routines, which aren't usually hanging out in the core code.

Oh, and in case you needed to see what I was building: a quick snapshot of my source. (http://www.mediafire.com/?ziyxmmm1mji) The C plugin routines still check for/specify fastcall and stdcall in places, but icc ignores these types when compiling 64bit. I thought it would be safe to leave them in if nothing was breaking as a result.

It is very cool to have ffms2 working somewhat correctly though, it can't be far off from having full blown functionality.

[kemuri-_9]

@kemuri-_9
Being the king of AMD PC's that you are, would you mind pulling a quick vertical resize test on any source to see if it's broken across the board for AMD users? If you could pull the latest binary (I uploaded a new one today), it'd be helpful to see if something funny is happening on the AMD side of things. You'll need it to properly test ffms2 anyway :).

har har, i have SWScale to resize with instead!
but yes, trying something like Lanczos4Resize(Width(last),Height(last)*2) is throwing
"Avisynth Unknown exceptions" exceptions here on my phenomII and athlon64 machines with this new build.

Giving a quick look at your loadCplugin vs the current one that checks stack corruption, I decided to just drop your LoadCPlugin function in as a replacement. As a result, I've got your ffms2.dll loading and the few tests I've run (various sources with some post processing effects, etc) have it running, with some oddities. Running a debug build of avisynth through through MSVS's debugger points to some code in ffms2.dll that is making illegal memory accesses. However, frames come through and appear correctly when running a release build of avisynth. There are no memory access violations thrown.

The oddity is that the 1st frame (frame 0) comes through as garbage. Moving around the source a bit, and then coming back to frame 0 has the frame rendering correctly. I figured I'd let you look it over, it may be my avisynth not initializing your plugin correctly. As you've probably gathered, I'm not too familiar with the plugin loading code of avisynth. My main focus has been on optimizing the calculation and memory heavy routines, which aren't usually hanging out in the core code.

I'm not experiencing either of these, though i can't manage to compile the source code you provided (I do have ICL 11.1.051) due to missing convert_a64.asm to make a debug build

[aegisofrime]

JoshyD, sorry for any confusion but the tests were on my Core 2 Duo machine, not my Phenom II machine. I do intend to test it on my Phenom II rig after my current encoding run is completed.

[JoshyD]

kemuri-_9
Whoops, I keep a CVS server running for my own code, forgot to add it to the tree. Here it is on it's own. (http://www.mediafire.com/?ntzygjnyiyy) Here's the source again, just to be on the safe side. (http://www.mediafire.com/?myogqtnmzno)

If you can point me to the code making AMD processors so unhappy, I'd really really appreciate it.

[turbojet]

Thanks levi for pointing to a newer RePAL version that outputs 25fps. About a year ago I had a blended pal->ntsc dvd that repal didn't handle all that well and tried SRestore to find worse results but maybe SRestore has improved since then. I agree repal should be low priority considering the low percentage of pal->ntsc sources.

Another filter that I've started to use lately is autocrop (http://forum.doom9.org/showthread.php?t=87602) which would be nice for avisynth64 and required for x264 input imo without it you need to depend on external programs to get crop values. Maybe something like --autocrop:width(none = no resize):resizer(lanczos default):mod(default 16) some examples with a 1920x1080 2.35:1 source:
--autocrop:1280 = lanczosresize(1280,544,0,132,0,-132)
--autocrop:1920 = --autocrop = crop(0,132,0,-132) (this should undercrop to mod set, since it's still higher quality then resizing)
if the source is 1920x1080 1.78:1 --autocrop:1920 = 1920x1080 with no crop/resize (mod8 input with no crop/resize would always be mod8 output, same for mod4/2?)
though something like --crop and --resize would be helpful in cases where autocrop doesn't work (which I haven't ran into yet). I really don't understand the need for --cli-filter prefix however.

Thanks JoshyD for tivtc I'll try it on some sources next week.

[Stephen R. Savage]

I just tried the new plugins, and I can say that all of them passed a quick test without issues. However, I do have a question regarding the "threads" parameter to some of these filters. Does the threads parameter get ignored on your Avs64 build, JoshyD?

Now if only we had nnedi2 :(

[JoshyD]

Nope, they'll thread themselves independently of the main avisynth dll. It's a bit of a balancing act that the user has to do to make all processors stay busy, while not choking them with too many threads. NNEDI2 would be nice, but I don't think we're going to see that anytime soon.

For now, I really want to figure out why AMD processors can't run the resizers in current binary . . . it's has to be something really simple that I'm missing.

[mavinashbabu]

Hi,


thread title says x86_64 so can i assume that it works on 32 bit versions of avisynth as well. can anyone confirm please.

Thanks,