SEt's Avisynth 2.5.8 MT compiled for *X86_64*, Latest Build 4/16/2010

[osgZach]

Script:


LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\dgdecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\telecidehints.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\fieldhint.dll")



function Preset0(clip c) {
#Name: Default
c
return last
}

DGDecode_Mpeg2Source("L:\D-Note\Ep 01\episode 01.d2v")



FieldHint(ovr="L:\D-Note\Ep 01\episode 01.d2v.fh.txt")


MT("""TempGaussMC_beta2().SelectEven()""",overlap=12,threads=2)


PresetClip0=Preset0()

PresetClip0.Trim(0,41023)



DClip = Decimate(cycle=5,quality=3,ovr="L:\D-Note\Ep 01\episode 01.d2v.dec.txt").assumefps(last.framerate)

##removed a bunch of Trim + Dclip.Trim statements from this line


aWarpSharp2(64,12,type=1)
Crop(0,4,-4,0)
Spline36Resize(720,480)

[Stephen R. Savage]

Stephen, thank you. But I wonder why plugin MVTools 2.5.10 works another way after its "translating" into x64? I suggest the default settings must stay constants. And the result of plugin's work must be the same. If behavior of plugins is changed then version number must be changed too with attached change list. If it is a bug then bug needs to be fixed.
I can't use this plugin because of its strange (unexpected) results. In my opinion about MVTools 32bit results is good and 64bit results is bad. Why it happens? Is it a bug?

Yes, it is most certainly a bug. Likely, some of the lesser-used functions were not correctly converted to 64-bit code, and they were overlooked because nobody was using them.

[osgZach]

Hmm wonder if I just made an important discovery..

I didn't have the newest mt_masktools sitting in my plugins64 directory, however I did have TWO different versions of mt_masktools, 2.5/2.6 I am wondering if a) I should have had two in the first place.. and

b) what the difference is between your newest released version simply titled "mt_masktools".

I deleted the older versioned ones, and stuck the singular "mt_masktools" in the plugins64 directory, so far no weird errors or anything. Hmm

[osgZach]

New access violation, this time it was decomb I think (never seen it show up before, but mvtools/masktools is decidedly absent from the dump).
This was a test using SetMTModes w/Huffyv12. I threw in TelecideHints as well to see how much faster I can increase encoding time (if TGMC'd frames and returned progressives from TFM do not look substantially different) , will try an MT.DLL variant with it next.

I suspect the decomb reference to be coming from TelecideHints (but am unsure) but wanted to make sure this wasn't an actual bug, although I believe TelecideHints probably just doesn't like Mode2
For reference this occured on: The new Intel Build, new "mt_masktools.dll", DGDecode 1.4.whatever,

VirtualDub crash report -- build 32706 (release-AMD64)
--------------------------------------

Disassembly:
7feef541360: 83c120 add ecx, 20h
7feef541363: 49 dec ecx
7feef541364: 83f820 cmp eax, 20h
7feef541367: 7db7 jge ef541320
7feef541369: 4c dec esp
7feef54136a: 8d15c0ecffff lea edx, [ffffecc0]
7feef541370: 49 dec ecx
7feef541371: 03d3 add edx, ebx
7feef541373: 49 dec ecx
7feef541374: 03c8 add ecx, eax
7feef541376: 49 dec ecx
7feef541377: 03d0 add edx, eax
7feef541379: 43 inc ebx
7feef54137a: ff24c2 jmp dword ptr [edx+eax*8]
7feef54137d: 666690 nop
7feef541380: 48 dec eax
7feef541381: 8d4940 lea ecx, [ecx+40h]
7feef541384: 48 dec eax
7feef541385: 8d5240 lea edx, [edx+40h]
7feef541388: 4d dec ebp
7feef541389: 8d40c0 lea eax, [eax-40h]
7feef54138c: 0f db 0fh
7feef54138d: 188280010000 sbb [edx+180], al
7feef541393: f30f6f42c0 movdqu xmm0, [edx-40h]
7feef541398: f30f6f4ad0 movdqu xmm1, [edx-30h]
7feef54139d: 49 dec ecx
7feef54139e: 83f840 cmp eax, 40h
7feef5413a1: 660fe741c0 movntdq [ecx-40h], xmm0
7feef5413a6: 660fe749d0 movntdq [ecx-30h], xmm1
7feef5413ab: f30f6f52e0 movdqu xmm2, [edx-20h]
7feef5413b0: f30f6f5af0 movdqu xmm3, [edx-10h]
7feef5413b5: 660fe751e0 movntdq [ecx-20h], xmm2
7feef5413ba: 660fe759f0 movntdq [ecx-10h], xmm3
7feef5413bf: 7dbf jge ef541380
7feef5413c1: 4c dec esp
7feef5413c2: 8d15b8010000 lea edx, [000001b8]
7feef5413c8: 4d dec ebp
7feef5413c9: 8bc8 mov ecx, eax
7feef5413cb: 49 dec ecx
7feef5413cc: c1e904 shr ecx, 04h
7feef5413cf: 49 dec ecx
7feef5413d0: c1e104 shl ecx, 04h
7feef5413d3: 49 dec ecx
7feef5413d4: 03c9 add ecx, ecx
7feef5413d6: 49 dec ecx
7feef5413d7: 03d1 add edx, ecx
7feef5413d9: 4d dec ebp
7feef5413da: 2bc1 sub eax, ecx
7feef5413dc: 49 dec ecx
7feef5413dd: c1e904 shr ecx, 04h
7feef5413e0: 0faef8 sfence
7feef5413e3: 43 inc ebx
7feef5413e4: ff24ca jmp dword ptr [edx+ecx*8]
7feef5413e7: 66666690 nop
7feef5413eb: 66666690 nop
7feef5413ef: 90 nop
7feef5413f0: 660f6f02 movdqa xmm0, [edx]
7feef5413f4: 660f6f4a10 movdqa xmm1, [edx+10h]
7feef5413f9: 660f7f01 movdqa [ecx], xmm0 <-- FAULT
7feef5413fd: 660f7f4910 movdqa [ecx+10h], xmm1
7feef541402: 4d dec ebp
7feef541403: 8d4080 lea eax, [eax-80h]
7feef541406: 660f6f5220 movdqa xmm2, [edx+20h]
7feef54140b: 660f6f5a30 movdqa xmm3, [edx+30h]
7feef541410: 660f7f5120 movdqa [ecx+20h], xmm2
7feef541415: 660f7f5930 movdqa [ecx+30h], xmm3
7feef54141a: 660f6f4240 movdqa xmm0, [edx+40h]
7feef54141f: 660f6f4a50 movdqa xmm1, [edx+50h]
7feef541424: 49 dec ecx
7feef541425: 81f8a8000000 cmp eax, 000000a8
7feef54142b: 660f7f4140 movdqa [ecx+40h], xmm0
7feef541430: 660f7f4950 movdqa [ecx+50h], xmm1
7feef541435: 660f6f5260 movdqa xmm2, [edx+60h]
7feef54143a: 660f6f5a70 movdqa xmm3, [edx+70h]
7feef54143f: 48 dec eax
7feef541440: 8d9280000000 lea edx, [edx+80]
7feef541446: 660f7f5160 movdqa [ecx+60h], xmm2
7feef54144b: 660f7f5970 movdqa [ecx+70h], xmm3
7feef541450: 48 dec eax
7feef541451: 8d8980000000 lea ecx, [ecx+80]
7feef541457: 7d97 jge ef5413f0
7feef541459: 4c dec esp
7feef54145a: 8d15d0ebffff lea edx, [ffffebd0]

Built on Aegis on Thu Dec 24 19:58:44 2009 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7600) []

RAX = 0
RBX = 5c0
RCX = 0
RDX = 22b97920
RSI = 2d0
RDI = 5c0
RBP = 7feef6217e0
R8 = 2d0
R9 = 200000
R10 = 0
R11 = 7feef540030
R12 = f0
R13 = 0
R14 = 0
R15 = 22b97920
RSP = 14a6c058
RIP = 7feef5413f9
EFLAGS = 00010246


Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'Veedub64'...

...writing address 0000000000000000.

Pointer dumps:

RDX 22b97920: 10111010 10100f10 10100f0f 10101111 0f0f1012 0f101110 0e0f1010 0f11100e
RSP 14a6c058: ef381a0d 000007fe ef621c60 000007fe 445c93e0 00000000 445c93e0 00000000
14a6c078: ef3907c9 000007fe 22b97920 00000000 00000000 00000000 00000000 00000000
14a6c098: 00000001 00000000 22b97920 00000000 00000000 00000000 000005c0 00000000
14a6c0b8: 03091fca 00000000 445c942c 00000000 ef38fd89 000007fe 445c93e0 00000000
RBP 7feef6217e0: ef380af0 000007fe ef3866f0 000007fe ef3866a0 000007fe ef3865c0 000007fe
7feef621800: ef386510 000007fe ef48cb60 000007fe ef3858a0 000007fe ef3857a0 000007fe
7feef621820: ef383ce0 000007fe ef383910 000007fe ef383420 000007fe ef383260 000007fe
7feef621840: ef383190 000007fe ef383140 000007fe ef382a90 000007fe ef381a60 000007fe
R9 00200000: 25048d48 00000010 082524ff 90000000 40171300 00000001 4016b540 00000001
R11 7feef540030: ef540724 000007fe ef5407ff 000007fe ef5408e2 000007fe ef5409c7 000007fe
R15 22b97920: 10111010 10100f10 10100f0f 10101111 0f0f1012 0f101110 0e0f1010 0f11100e

Thread call stack:
7feef5413f9: avisynth!avs_at_exit [7feef370000+11c9c0+b4a39]
7feef381a0d: avisynth!avs_clip_get_error [7feef370000+10280+178d]
7feef38fd89: avisynth!avs_release_video_frame [7feef370000+1b530+4859]
03091b3d: fieldhint!00001b3d
76ab56c0: kernel32!lstrcmpi [76ab0000+5680+40]
7feef3835a0: avisynth!avs_clip_get_error [7feef370000+10280+3320]
7feef38cf09: avisynth!avs_release_video_frame [7feef370000+1b530+19d9]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
02ba138f: telecidehints!0000138f
7feef38156c: avisynth!avs_clip_get_error [7feef370000+10280+12ec]
7feef38e504: avisynth!avs_release_video_frame [7feef370000+1b530+2fd4]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7fefcf918aa: KERNELBASE!ResetEvent [7fefcf90000+18a0+a]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
76d1d1a8: ntdll!CsrAllocateMessagePointer [76cd0000+4cf80+228]
7fefcf910ac: KERNELBASE!WaitForSingleObjectEx [7fefcf90000+1010+9c]
7feef3e21b9: avisynth!DllGetClassObject [7feef370000+25d80+4c439]
76d1d0a2: ntdll!CsrAllocateMessagePointer [76cd0000+4cf80+122]
7feef38e504: avisynth!avs_release_video_frame [7feef370000+1b530+2fd4]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7fefcf918aa: KERNELBASE!ResetEvent [7fefcf90000+18a0+a]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
02b63f70: decomb!AvisynthPluginInit2 [02b60000+1000+2f70]
7feef38156c: avisynth!avs_clip_get_error [7feef370000+10280+12ec]
7feef38fd89: avisynth!avs_release_video_frame [7feef370000+1b530+4859]
02b618e2: decomb!AvisynthPluginInit2 [02b60000+1000+8e2]
7feef3835a0: avisynth!avs_clip_get_error [7feef370000+10280+3320]
76d1d238: ntdll!CsrAllocateMessagePointer [76cd0000+4cf80+2b8]
7feef38156c: avisynth!avs_clip_get_error [7feef370000+10280+12ec]
7feef38cf81: avisynth!avs_release_video_frame [7feef370000+1b530+1a51]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
02b617b8: decomb!AvisynthPluginInit2 [02b60000+1000+7b8]
7feef38e504: avisynth!avs_release_video_frame [7feef370000+1b530+2fd4]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7fefcf918aa: KERNELBASE!ResetEvent [7fefcf90000+18a0+a]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7feef3de282: avisynth!DllGetClassObject [7feef370000+25d80+48502]
7feef38e504: avisynth!avs_release_video_frame [7feef370000+1b530+2fd4]
76d21c58: ntdll!RtlAllocateHeap [76cd0000+51b70+e8]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7fefcf918aa: KERNELBASE!ResetEvent [7fefcf90000+18a0+a]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7feef3e21b9: avisynth!DllGetClassObject [7feef370000+25d80+4c439]
7feef526b0f: avisynth!avs_at_exit [7feef370000+11c9c0+9a14f]
7feef38e504: avisynth!avs_release_video_frame [7feef370000+1b530+2fd4]
7feef523a43: avisynth!avs_at_exit [7feef370000+11c9c0+97083]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7fefcf918aa: KERNELBASE!ResetEvent [7fefcf90000+18a0+a]
7feef39007e: avisynth!avs_release_video_frame [7feef370000+1b530+4b4e]
7feef381688: avisynth!avs_clip_get_error [7feef370000+10280+1408]
7feef3e0c66: avisynth!DllGetClassObject [7feef370000+25d80+4aee6]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]
7feef3e0c50: avisynth!DllGetClassObject [7feef370000+25d80+4aed0]

-- End of report




LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\dgdecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\telecidehints.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\fieldhint.dll")



function Preset0(clip c) {
#Name: Default
c
return last
}
SetMTMode(5,0)
DGDecode_Mpeg2Source("L:\D-Note\Ep 01\episode 01.d2v")



FieldHint(ovr="L:\D-Note\Ep 01\episode 01.d2v.fh.txt")



SetMTMode(2)
TelecideHints(TempGaussMC_beta2().SelectEven())

PresetClip0=Preset0()

PresetClip0.Trim(0,41023)



DClip = Decimate(cycle=5,quality=3,ovr="L:\D-Note\Ep 01\episode 01.d2v.dec.txt").assumefps(last.framerate)

#and obligatory omitted Trim + Dclip.Trim statements.

[JoshyD]

@osgZach

The difference between the two plugins is actually the version of avisynth.h that they were compiled with. My version latest version definitely go changed to use my version of avisynth.h, I think the previous versions were built using avisynth 2.6's pre-release header that was packaged with the binary/source. The titling of the plugin to just "mt_masktools.dll" was just for simplicity. The 32 bit versions of 25 and 26 are meant for use with avs2.5 or avs2.6. That's about it, I think the last compile that you downloaded is solid, but let me know any problems that may arise. Try to keep only a single version of the plugin in the auto-load directory, I can't remember how avisynth does it's loading, but it's something along the lines of the last one loaded with the function name will be used.

Your new crash doesn't look like any code I've written . . . at least lately or extensively. It looks like a memory copy, maybe it is related to SetMTMode, keep me updated on the crashing.

@Stephen
What exactly is the problem with MVTools? The one labeled "64" plays back smoothly for me, whereas the other version labeled "32" displays some strange motion from frame to frame. Running it on my machine gives the same results, so, should the end result shouldn't be displaying a blurring of two frames to create the new frame rate?

Edit: I think I found the problem, it should be an easy fix.

@levi
I can LanczosResize just fine . . . but can you you Lancsoz4Resize? If so, then that's quite the strange problem.

[osgZach]

For the time being I'll stick with MT("filter") but will do tests on new releases of Avisynthx64 and MV/Masktools x64 releases from time to time and provide crash dumps each new version if that's OK with you. I just don't want to end up littering the thread with crash dumps that may be the same and nothing related to your work.

Huffyv12 seems just as fast as UT was and adaptive tables keeps the size very competitive so I'm going back to that now that I have access to it again. Hopefully it will provide some stability to the process as well.. I'll probably go back and recompare DGDecode 1.5.8 performance again as well.. If you can find time to reimplement the iDCT's that would be great I guess, but for now I'm using SSE2 with no problems.

I haven't done extensive testing with TIVTC as it pretty much crashed on me, but I don't recall why. Hopefully new versions will come along in the future though, with some improvements. Haven't forgot about the web site either, I started experimenting with a layout so the URL didn't 404 all the time ;) but I haven't had the time to do a proper sit down session and really work at it.

Also I recommend anyone stay away from UT codec until AVIsynth x64 and most plugins are stable. I think there may be stability issues, but I can definitely confirm, much to my expensive chagrin, that it is not suitable for lossless workfiles based on VFR. If you go back and re-encoding with other postprocessing (in my case after a lengthy TGMC IVTC) your VFR file will get screwed up for some reason. I ended up with 37 files that even when muxed with timecodes did not play back properly and had huge frame skipping / jerky playback issues. Huffyuv/yv12 seem to be safe based on a quick test I did, however, so ffdshowx64 is imperative for those doing lossless intermediate files under a pure x64 encoding chain.

I got my version from http://x264.nl - not sure if its the latest but I've been skeptical to install the Tryouts site version as it seems to be in perpetual "beta" state, or the wording is just in a way that scares me off maybe.

[squid_80]

A word of warning, virtualdub's x64 disassembler has some bugs and the code might not be accurate (as demonstrated by the supposed fault on a dec instruction).

[AlekseiV]

Well, a simple "avisource("blah.avi")" errors out because it appears that megui is looking for a 32-bit codec, which would require a 32-bit install of ffdshow, which I'm a little wary to do, as I don't know how it will handle two versions of ffdshow installed on the same system.Pretty sure a 32-bit ffdshow will only interact with 32-bit players/whatevers, just how x64 only interacts with x64. I have them both running without any noticeable problems.

[levi]

@levi
I can LanczosResize just fine . . . but can you you Lancsoz4Resize? If so, then that's quite the strange problem.

I removed everything avisynth related & re-installed. Now the issue is gone. :confused:


I have another oddity:

When encoding with the 64 bit xvid_encraw from http://members.optusnet.com.au/squid_80/ I get inaccurate FPS reported (it's reported correctly @ 12fps with the 32 bit version):


xvid_encraw64.exe -imin 2 -imax 4 -pmin 2 -pmax 7 -bmin 2 -bmax 10 -bvhq -vhqmode 4 -qtype 0 -progress 300 -stats -i "my.avs" -pass1 "my.xvid.stats" -bitrate 1004 -threads 1



Trying to retrieve width and height from input header
xvid [info]: Avisynth detected
xvid [info]: Input colorspace is YV12
xvid [info]: Input is 512 x 384, 23.976fps (24000/1001), starting from frame 0
xvid [info]: Number of frames to encode: 124472, Bitrate = 1004kbps
xvid [info]: xvidcore build version: xvid-1.2.2
xvid [info]: Bitstream version: 1.2.2
xvid [info]: Detected CPU flags: ASM MMX MMXEXT SSE SSE2 TSC
xvid [info]: Detected 4 cpus, using 1 threads.
124201 frames( 99%) encoded, 310.56 fps , Average Bitrate = 1200kbps
Tot: enctime(ms) =400343.00, length(bytes) = 777934630
Avg: enctime(ms) = 3.22, fps = 310.92, length(bytes) = 6249, psnr y = 0.00,
psnr u = 0.00, psnr v = 0.00
I frames: 632 frames, size = 26343/16649129, quants = 2 / 2.00 / 2
P frames: 43223 frames, size = 13059/564455603, quants = 2 / 2.00 / 2
B frames: 80617 frames, size = 2441/196829898, quants = 4 / 4.00 / 4



It gets worse with the one from http://www.xvidvideo.ru/xvid-video-codec/xvid-1-2-2-x86-x64-stable-release-vaq.html

The xvidvideo.ru version cannot even detect the avisynth input information at all. The reported FPS is also inaccurate.

Any ideas on how to get the FPS reporting properly?

[aegisofrime]

JoshyD, just want to provide feedback for my libmio5 *something* thingie. Downloading your AMD build fixed it somehow :) Thanks for the great work!

[squid_80]

The xvidvideo.ru version cannot even detect the avisynth input information at all. The reported FPS is also inaccurate.

Any ideas on how to get the FPS reporting properly?

It's an xvid_encraw issue, it only reports the time taken to encode the frame rather than including the processing time taken to fetch the input frame. It was fixed in my 32-bit build but the source code is buried in storage.
The avisynth information was also something extra that I added to my build and is not part of the code from xvid's cvs.

[levi]

I'm sure you would have offered if you really wanted too, but I'll go ahead & ask... any chance you could dig up that source? :)

[osgZach]

After some further comparisons between DGdecode 1.4.6 & 1.5.8 I'm seeing 1.5.8 as falling between 10 - 20 minutes slower (averaing 12m - 16m slower). Is that consistent with other peoples findings?
Seems stable and everything, so that's good to know. Will probably stick with it despite the minor slowdown. Not sure how that quantifies into FPS but I'm guessing some kind of fraction of an FPS.

[MuLTiTaSK]

testers needed -> Zathor x64 MeGUI build (bit.ly/9EpprX)

[Delerue]

I have just tested x64 versions of Avisynth and MVTools with XP x64 and both worked flawlessly. MVTools had a great performance boost. Thanks a lot, JoshyD.

[Stephen R. Savage]

Any luck with that GradFun2DB? It's the last plugin holding be back from using Avs64 as my main encoding platform.

[osgZach]

I haven't tested it with option calls, but I have invoked it without error just to see if it would load.

http://www.mediafire.com/?gogdnjtzinf

If you didn't notice it on the first page.

[turbojet]

I decided to make an install script for both amd/intel (http://www.mediafire.com/?mmo2dz3ozmx) since I'm now using it on both platforms and got sick and tired of keeping track of 2 dll's. Just thought I'd post it unless anyone else was in the same situation.

Edit: Also with resizing to anything I'm getting this error in x264 r1510:

x264 [warning]: non-strictly-monotonic pts at frame 1 (0 <= 0)
x264 [warning]: non-strictly-monotonic pts at frame 2 (0 <= 1)
x264 [warning]: non-strictly-monotonic pts at frame 3 (0 <= 2)
x264 [warning]: too many nonmonotonic pts warnings, suppressing further ones

There is no error when not resizing or with avisynth 2.6 resizing.

[VincAlastor]

thanks a lot for gradfun2db x64!!! now i just have to wait for a x64 version of dg nv tools :-)
thank you guys for your hard work

[JoshyD]

@turbojet
I'm not really sure what that error means . . . I'm aware of monotonic functions in mathematics, but how this relates to x264 and video compression, I'm not really sure. I get the same error, I think I may have to pop over to doom10 and ask there, if you haven't already. I'll comb through the source and see if I can find what generates the error in the first place. With older builds of x264, no complaints, so this is must be a new error. Unfortunately, most of the code for the project currently is 64bit specific, so I can't build a 32bit version of my source to test against.

For me, the resulting video file is all sorts of bizarre as well. The frame rate doesn't appear to be written to the file correctly, the encoder does it's thing reporting the correct rate, the resulting file has the correct fps info, but playback is sporadic and looks like it's ~2x faster than what it should be. Older builds produce correct results.

I've got a test release that I want to get out today that should be platform agnostic, so you don't have to worry about managing two dll's. Sorry it took so long to finally get around to it.

@Stephen
As far as the GradFun2DB issues, I haven't gotten a chance to sort it out, but I'll take a look at it today.

Edit: Glaring code mistake on my part, fixed it up, and it now produces the same output as the 32bit version. Try it out and let me know if there are still any issues.

[Mr VacBob]

It means that input frame timestamps aren't strictly increasing (in monotonic order). This can be caused by, well, nearly any kind of bug in the x264 input module being used - in this case it looks like the input pts is always 0.

You can fix it with some combination of --force-cfr and --fps, but I forget how that works.

Speaking of x264, it would be nice if you adopted the ISC-license yasm system it uses for i386/x86-64/win64 asm, it's all portable and stuff.

[Stephen R. Savage]

Thanks for updating GradFun2DB. I hope that you can get the remaining IDCTs in DGDecode sorted out, but with this, I can finally start encoding in 64-bit.

Edit: I know this is really "asking the impossible," but could you think of any way to allow calling 32-bit plugins within a 64-bit Avisynth instance? It doesn't have to be efficient or even elegant. Even just internally spawning an Avs32 instance and copying frames from it would be acceptable. The reason I ask is that working without nnedi2 is just a massive drag.

I was thinking something like this hack would work, if not too many 32-bit plugins were used, and they were called infrequently:


Load32Plugin() opens a win32 DLL and scans it for Avs32 functions.
Whenever a function associated with a 32-bit DLL is called, export the script environment referenced by the call to shared memory. This would include clips and the value of variables.
An Avs32 instance is created in another process. This process loads a helper ImportFrom64() source that reads the variables exported by Avs64.
The 32-bit subprocess has the function ExportEnvironment() which somehow transfers variables and clips to shared memory.
Avs64 reads the data exported by Avs32 and updates the script environment.

Another idea:

Implement the function Eval32() that runs one command in an Avs32 environment and reads the value of "last" back. The last clip would be exported via shared memory, as well as any other clips and variables the user specifies.

Would the overhead from these suggestions be unacceptable?


I am thinking of one specific usage case for this:

You want to use TGMC with nnedi2. You posit that if the 32-to-64 memory transfer can occur fast enough, the performance benefits of using 64-bit mvtools outweigh the overhead of the memory copy.

I know that something similar is currently implemented with TCPSource/TCPDeliver (although it is not implemented in current Avs64), but it is not transparent to the script.

[kemuri-_9]

Edit: I know this is really "asking the impossible," but could you think of any way to allow calling 32-bit plugins within a 64-bit Avisynth instance? It doesn't have to be efficient or even elegant. Even just internally spawning an Avs32 instance and copying frames from it would be acceptable. The reason I ask is that working without nnedi2 is just a massive drag.

the best you'll get is TCPDeliver

[Stephen R. Savage]

the best you'll get is TCPDeliver

I know of TCPDeliver, but a fatal flaw is that you can not call it more than once, nor can you use it in combination with TCPSource to shuffle data. It also requires the user to manually load any external scripts.

[kemuri-_9]

I know of TCPDeliver, but a fatal flaw is that you can not call it more than once, nor can you use it in combination with TCPSource to shuffle data. It also requires the user to manually load any external scripts.

Script A (x64)
...
TCPDeliver


Script B (x86)
TCPSource
...
TCPDeliver


Script C (x64)
TCPSource
....

-> x264 x64

to do something like you're wanting this would be more reliable to be done as a plugin that spawns avs x86 a seperate process in some fashion to pipe data to and from it in some fashion.
by no means is this a small or clean task.

[turbojet]

@turbojet
I'm not really sure what that error means . . . I'm aware of monotonic functions in mathematics, but how this relates to x264 and video compression, I'm not really sure. I get the same error, I think I may have to pop over to doom10 and ask there, if you haven't already. I'll comb through the source and see if I can find what generates the error in the first place. With older builds of x264, no complaints, so this is must be a new error. Unfortunately, most of the code for the project currently is 64bit specific, so I can't build a 32bit version of my source to test against.

For me, the resulting video file is all sorts of bizarre as well. The frame rate doesn't appear to be written to the file correctly, the encoder does it's thing reporting the correct rate, the resulting file has the correct fps info, but playback is sporadic and looks like it's ~2x faster than what it should be. Older builds produce correct results.

After further investigation it wasn't just any resize that gives the warning. 1280x712, 640x360, 1920x720 didn't have the warning but these did: 1280x720, 1280x728, 1280x1080, 1280x960, 960x720. I tried the SSE dll you had amd users try a few weeks ago and I couldn't reproduce the warning. I'm not getting playback issues with MPC-HC with DXVA, ffdshow, or coreavc but I've only tried <10000 frame clips.

I've got a test release that I want to get out today that should be platform agnostic, so you don't have to worry about managing two dll's. Sorry it took so long to finally get around to it.

No rush, it only took me 5 minutes to add the 2 lines, so no harm.

Just out of curiosity was porting x86 to x64 avisynth tedious and/or difficult for someone who has never ported? I'm asking because I've been contemplating trying it out with 2.60 because I'd like to get back the performance lost in the MT hack. Official 2.58 build is about 8% faster then set's 2.58 MT build with my light scripts that include only crop, resize, ivtc, fast deinterlacing.

[squid_80]

I know this is really "asking the impossible," but could you think of any way to allow calling 32-bit plugins within a 64-bit Avisynth instance? It doesn't have to be efficient or even elegant. Even just internally spawning an Avs32 instance and copying frames from it would be acceptable. The reason I ask is that working without nnedi2 is just a massive drag.

I was thinking something like this hack would work, if not too many 32-bit plugins were used, and they were called infrequently:


Load32Plugin() opens a win32 DLL and scans it for Avs32 functions.
Whenever a function associated with a 32-bit DLL is called, export the script environment referenced by the call to shared memory. This would include clips and the value of variables.
An Avs32 instance is created in another process. This process loads a helper ImportFrom64() source that reads the variables exported by Avs64.
The 32-bit subprocess has the function ExportEnvironment() which somehow transfers variables and clips to shared memory.
Avs64 reads the data exported by Avs32 and updates the script environment.

Another idea:

Implement the function Eval32() that runs one command in an Avs32 environment and reads the value of "last" back. The last clip would be exported via shared memory, as well as any other clips and variables the user specifies.

Would the overhead from these suggestions be unacceptable?


I did attempt this, the overhead was minimal since the filters could blit frames directly into the shared memory regions. But the major issues were handling audio as well as video, and exporting multiple clips for input to a single proxy function.

[osgZach]

Personally I'm of the opinion audio support is not a necessity. There are a million ways to get your audio into the final clip. If supporting video only makes the last problem that much easier to handle, I say go for it. It's not the end of the world, right?

[JoshyD]

Just out of curiosity was porting x86 to x64 avisynth tedious and/or difficult for someone who has never ported? I'm asking because I've been contemplating trying it out with 2.60 because I'd like to get back the performance lost in the MT hack. Official 2.58 build is about 8% faster then set's 2.58 MT build with my light scripts that include only crop, resize, ivtc, fast deinterlacing.

Tedious, yes. Difficult, depends.

Tedious because the actual code base is quite large, and some parts of it have been hanging around for quite some time. For example, _control87() is used in one of the init functions for scripts. 64bit apps don't want you changing the fp state and such. Then, there's any parts of the code that rely on pointers being the same size as integers. This get's sticky at times. I remember missing a few instances when some assembly was indexing an array of pointers. They're not in 4 byte chunks anymore . . . that's an easy one to miss at first glance.

The resizing code actually lucks out that memory created local to a function will never have an address >32bits. The resize code stores address offsets in an array of integers, along with coefficients for the corresponding address. If the address were larger than an integer, it would have never worked out under x64.

The most difficult part for me was just learning assembly, and more specifically 64bit function calling and returning (where's my data and why did the calling function write 32bits of it, but allocate 64?). There's inline assembler littered throughout the source, and the resize functions use snazzy dynamically compiled assembly. To be competitive in this aspect, I statically generated most cases that would arise when dynamic code was going to be used. To ease in the static generation, you'll need to learn the ins and outs of writing macros and such to easily create x functions from one piece of code.

Then there are instances where you're going to have to pay special attention if a negative value is ever used. If you want to manipulate an address using a negative offset, you have to make sure to sign extend it first. There's a decent number of quirks that tripped me up on the first pass through.

Also, seriously, if you want to port the whole code base, set up some sort of version control. My first attempt was totally scrapped, then I set up a cvs server on my own machine, and it has since really cut down on the amount of time spent searching for the crazy little syntax change I forgot about.

A note on the inline assembler, you'll need Intel's compiler to even think about compiling the source without re-writing all the inline assembler in their own modules.

With some decent C++ background and some understanding of computer organization, it wouldn't be all too difficult to port the 2.6 code as well. I was actually considering doing this one of these days. The majority of the code has been merged into the head branch of the project, and I've been building it for a while. I'm going to hazard a guess that it's close to being stable. If you've got some free time, I've got a google code project setup finally, we could start a 2.6 branch there. If you prefer working on it on your own, the source is all there, and it's a decent starting point for converting 2.6. I guess just let me know.

[osgZach]

Teaming up is always nice I say. Delegate out the easy stuff to the n00b and then work on the hard stuff yourself, you'll be done in no-time ;)

Always nice to be working off stable code too.. So I'd say at least wait for an official release before releasing a x64 port to the public, though.

[squid_80]

The resizing code actually lucks out that memory created local to a function will never have an address >32bits. The resize code stores address offsets in an array of integers, along with coefficients for the corresponding address. If the address were larger than an integer, it would have never worked out under x64.


Please tell me you didn't leave the code like this. If you did, you're setting up an inevitable point of failure. Avisynth is a library and should not make *any* assumptions about the location of the stack.

[turbojet]

The porting sounds beyond my knowledge. I was thinking it was just translating certain calls but it sounds like new code has to be produced. The only code I've ever produced in c++ is calling a couple external programs.

Ideally what I'd want to happen at some point is avisynth being a multiple installer that if windows 64 is found both 32 and 64 bit are installed/registered otherwise only 32 bit avisynth. Like haali and coreavc currently do. That requires JoshyD to work with IanB, Wilbert, etc. They might welcome a 64 bit 'translator' have you consider contacting them about it? Or would you rather avisynth64 stay a separate project?

Concerning avisynth alpha versions, while their probably is bugs introduced from the last release version. From my experiences since 2.08 is there usually many more bugs fixed then introduced in the alphas. After browsing the thread I see only one known outstanding bug (mpeg2source(cpu=1+)) while many bugs have been fixed from 2.58.

[Poutnik]

Are there plans for porting DCTFilter to x64 ?
I was told plugins needed for MCTemporalDenoise by LaTo were ported,
but I noticed DCTFilter required by Didée's Deblock QED was not.

Didée's post in MCTemporalDenoise [v1.4.08 - Update 2010/01/30] thread (http://forum.doom9.org/showthread.php?p=1387595#post1387595)

Edit: BTW, with deblock=false, GPU=true and for testing commented out all SetMTmode commands,
I was able to load MCTD calling script, but Vdub64 crashed after few frames.
( latest 64b avisynth and plugings on MAR 30th, Vdub 1.9.8 stable )

IsInterlaced = false
mpeg2deblock = 0
AudioDub(DGDecode_mpeg2source("D2Vfile", info=3, cpu=(mpeg2deblock==1)? 4:0), wavsource("wavfile")).ColorMatrix(d2v="D2Vfile",interlaced=IsInterlaced, threads=0)
MCTemporalDenoise(sigma=5,settings="low",GPU=true,Deblock=false)

No particular crash info, usedto see at VDub, maybe later.
Edit2" avs2avi 64bit lasts longer, but not much.

[Andrey /MAG/]

@JoshyD
Thank you for last build of 64-bit MVTools2 (29 march).
But the results of my script are still differ from results of 32-bit build: new frames are still blended, not motion compensated. See details in http://www.sendspace.com/file/4u8s0t
How can I get the same results?
When the source code will be accessible?

[osgZach]

Not sure where the problem is exactly.. But I got the updated Avisynthx64, EEDI2, and Mvtools

Opened an AVS and started to encode and immediately got an Avisynth read error, Vdubx64 subsequently decided to crash whenever closing an AVS file.

I reverted Avisynth to 3-19 Intel build.. that seemed to stop the crashing on closing an AVS in Vdub..

Then I reverted to the older mvtools2, seems the newer one was causing the read errors.

[JoshyD]

@squid_80
Addresses local to your program can never be more than 32bit offsets relative to the instruction pointer. This is due to the implementation of x86-64. It is possible for a process to access the full 64 bit address space, but this is generally done by a process running at the kernel level, using special instructions. AMD docs refer to these as SIB memory accesses. The memory reference starts at 0 and works up computing the address with scale and index bits from the instruction. The virtual address space is currently limited to 48bits because of the original implementation of the Athlon64. Furthermore, because this is a Windows program, a single process cannot "see" memory beyond 32bit addresses, unless allocating memory with VirtualAlloc() or its sibling functions (Windows specific, non C standard allocation functions, and we're tethered to Windows inherently because of vfw reliance). The PE32+ specs are sketchy at best, MS claims that the HEAP RESERVE and HEAP COMMIT fields have been widened beyond 32bits. MS docs list these as just "widened," but not to any specific bit length. Compilers still write these values in 32bit lengths. Basically, if your program is not the OS, good luck getting into addresses that are beyond a 32bit value from your instruction pointer. All of your physical data references will be 32bits, and translated by the OS to a virtual address. Instruction data does hang out in the upper address space, though. Windows specifically loads 64 bit executables at addresses beyond what a 32bit process can use (>4GB). Anyhow, my point is, for the foreseeable future, writing the memory offset into an int array won't cause massive failures.

@Poutnik
You are one step ahead of me there. I've been wanting to test that script out, but haven't had the time. My quick response is that I've already found some silly bugs in my release yesterday, and should have a quick fix out soon (today-ish).

My longer response is I'll get around to porting DCTFilter, and then test the script well, as it's one that I personally have had great use for. With a little time, I should be able to get it running well.

@Andrey /MAG/
I did look at this quirk briefly, and think I found the source of the problem, but didn't have time to properly fix it. An older version of my source is available here, (http://www.mediafire.com/?yuzzwy3m1y1) as well as buried somewhere in the first post. The changes between that source and the current build don't relate to your blending bug, I've only really changed the memory copy routines, x264 functions ported directly from the x264 repository, and removed CPU checking for mmx/sse. I'll get the current source online so you can browse at your leisure. Which reminds me . . .

@osg_Zach
Any updates on getting a quick web page up? Something to just link the plugins/sources would be useful now that there's some interest in seeing them. The first post of this thread can only get so long ;). And finally,

@Turbojet
I haven't really considered contacting them, I wouldn't mind translating the project officially. I think it does require an official branch, though. There are so many #ifndef _AMD64_ instances in my source, that I eventually gave up on trying to compile my source both 32bit and 64bit for error checking. I'm also guessing that they'd want to see a final source that can be compiled by anyone, not just those with the Intel's C++ compiler, which means dismantling some functions that rely on inline assembler (look at the yv12 conversion routines, yikes!).

I was also afraid of coming to them with my source looking as unpolished as it is. It really does look terrible in some places. When I began the port, I wanted to get a working binary ASAP, which meant bad hack and slash coding on many occasions. If I can get the time to polish it up so it doesn't look like I was coding-impaired, I'll see what they think. Sidenote: Remember that Jeremy Duncan character? A lot of people thought I was him when I originally posted my work. I hope I'm not still carrying that stigma around . . . with the number of *working* plugin ports I've done, (aWarpSharp, MaskTools, and MVTools2 (working except for the blended frame issue) were all fairly complex ports) I hope I'm the real me in most people's opinions.


WARNING
The 3/29/2010 posted binaries for EEDI2, MVTools, and Avisynth64 are *not* stable. Fixed and linked on the first post.

[osgZach]

Responded to your PM. I have been dragging my feet on the web site thing, so I apologize for that. I had a tiring work weekend, and am working my second job Mon - Fri this week, plus helping my first job on Fri after that. So just been sleeping and diddling around..

I am looking into some ideas right now. I want to try and see what I can do with Word Press or another simple blog like that, so you can have a simple posting based site (and FTP access to upload/update Builds and Plugins). This way I don't have to worry about keeping the site up to date, and you don't get stressed that I haven't updated it ;)

The most you'd have to learn is how to login, make posts, and write a hyperlink to a file (assuming there isn't an easy button for that in wordpress), and update said hyperlink..

[squid_80]

@squid_80
Addresses local to your program can never be more than 32bit offsets relative to the instruction pointer. This is due to the implementation of x86-64. It is possible for a process to access the full 64 bit address space, but this is generally done by a process running at the kernel level, using special instructions. AMD docs refer to these as SIB memory accesses. The memory reference starts at 0 and works up computing the address with scale and index bits from the instruction. The virtual address space is currently limited to 48bits because of the original implementation of the Athlon64. Furthermore, because this is a Windows program, a single process cannot "see" memory beyond 32bit addresses, unless allocating memory with VirtualAlloc() or its sibling functions (Windows specific, non C standard allocation functions, and we're tethered to Windows inherently because of vfw reliance). The PE32+ specs are sketchy at best, MS claims that the HEAP RESERVE and HEAP COMMIT fields have been widened beyond 32bits. MS docs list these as just "widened," but not to any specific bit length. Compilers still write these values in 32bit lengths. Basically, if your program is not the OS, good luck getting into addresses that are beyond a 32bit value from your instruction pointer. All of your physical data references will be 32bits, and translated by the OS to a virtual address. Instruction data does hang out in the upper address space, though. Windows specifically loads 64 bit executables at addresses beyond what a 32bit process can use (>4GB). Anyhow, my point is, for the foreseeable future, writing the memory offset into an int array won't cause massive failures.


I'm not buying it. It's true that static data is accessed using 32-bit relative offsets, but you're not talking about static data. If ALL memory accesses were restriced to 32-bit offsets you would instantly get failures, since the kernel functions live at the top of memory and by default programs are at the bottom. DLLs like avisynth can be anywhere in the middle, depending on what their base address is set to (and in later OSes the base address is effectively ignored if ASR is used).

64-bit processes most certainly can see beyond 32-bit addresses, you can use a stack up there or you can change the default behaviour of windows to start dynamic allocation from the top instead of the bottom(HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management\AllocationPreference). Avisynth is already susceptible to getting memory allocated above 4GB due to the way it uses excess memory for caching.
If you want this code to be fully 64-bit compatible, it must treat addresses as 64-bits. Anything else is a time bomb.

[Poutnik]

MCTD stability in Vdub or AVS2AVI with AVS / EEDI2/ MVtools updates from 31/3 is lets say 30 % better :-) Tried both GPU=true/false

[Poutnik]

I performed AVS2AVI benchmark of avs containing basically just SetMTmode(2) and Mpeg2source.
Old AVS 32 MT 2.57.5 gives 180 FPS
New AVS 64 MT 2.5.8 (from 31/5, DGdecode v1.5.8) shows just 80 FPS, so less than 50%.
But it could be due mentioned some missing idct algos. ( used default )

Setmemorymax was set for both to 768, but changing it does not have big impact.

[turbojet]

It's only about a 5% speed decrease here using x264 but I also see about 3% decrease with 2.58 over mt 2.57. I didn't use avs2avi because many builds spit out exotic fps numbers, only squid_80's x86 builds give proper fps. I do notice the latest mt x64 build is using 2 theads per core if setmtmode is set to auto detect which could hurt performance.

That said avisynth MT is mostly placebo and in most cases it slows the script down significantly, like it does in this case. Even using an mt avisynth.dll slows you down. SetMTmode() and MT() are useful with heavy filters but even then it's usually not significantly faster then non-mt avisynth. I'm crossing my fingers a non-mt avisynth 2.60 comes out then everyone can start seeing the 15-20% speed increase like what we see with this build over 2.58 mt. However currently this avisynth is lucky to get a 5% speed increase over 2.60 and sometimes it's slower. 2.57 x64 is often a little faster then this build except not when resizing. In conclusion, 2.58 mt seems to be the slowest avisynth build among 2.57 - 2.60 in my tests.

It's looking like avisynth devs have been considering adding MT for some time but in a much more efficient way, hopefully it won't slow it down.

JoshyD: Have you come across anything that might be causing the x264 pts warning when resizing to certain resolutions? I haven't seen any weird playback issues but concerned that you did and have reverted back to using older x64 builds that don't have the warning. Also I hope you don't take my above comments disgcouragingly, I greatly appreciate you taking on avisynth x64. Just after a little testing with various builds I was disappointed in the performance decrease of avisynth mt, even with ivtc, deinterlacing, denoising, nothing real heavy though. But specifically 2.58 mt which is the slowest and happens to be what you built off of.

[osgZach]

MT is placebo? Tell that to my cut-in-half encode times.

It's one thing to say there are instances where it may not be the best performing option, and others where it really helps increase performance, but you can't have it both ways.

[turbojet]

What I mean is mostly placebo is people see multithreading avisynth and assume it's faster but in most cases it's actually slower. I don't know of a gui that makes use of it, it's not a complicated addition but it slows down all the functions they provide.

The assumption isn't necessarily wrong because almost all multithreaded programs have significant performance increase over their single threaded counterpart.

By cutting encoding time in half you must be using some very heavy filters, what exactly are you using?

[levi]

In my testing, I have found that performance can vary greatly depending on:

The complexity of the script
Using MT vs. not using MT
Using setmtmode properly
Choosing the number of setmtmode / MT threads (I have never had good luck with threads=0)
Choosing the number of encoder threads (I have never had good luck with threads=0)

I keep a log of my fastest script & threading combinations.

I have also seen hardware play a role in encoding speed (enabling Hyperthreading in the bios actually slows my encodes by about 10%).

Generally speaking, in my experience MT seems to be much more effective with x264 encodes vs. xvid (and multithreading in general has very low effectiveness in xvid).

Basically, I would caveat any recommendation to use it by saying "Your Mileage May Vary". There are certainly cases where MT does nothing to help.

However, to say MT avisynth is a placebo - I must disagree with that as that has not been my experience.

[JoshyD]

It's only about a 5% speed decrease here using x264 but I also see about 3% decrease with 2.58 over mt 2.57. I didn't use avs2avi because many builds spit out exotic fps numbers, only squid_80's x86 builds give proper fps. I do notice the latest mt x64 build is using 2 theads per core if setmtmode is set to auto detect which could hurt performance

Using two threads per core is strange, I'll go back and look at the threading auto-detection, I had never really run into this problem.

Even using an mt avisynth.dll slows you down. SetMTmode() and MT() are useful with heavy filters but even then it's usually not significantly faster then non-mt avisynth. I'm crossing my fingers a non-mt avisynth 2.60 comes out then everyone can start seeing the 15-20% speed increase like what we see with this build over 2.58 mt. However currently this avisynth is lucky to get a 5% speed increase over 2.60 and sometimes it's slower. 2.57 x64 is often a little faster then this build except not when resizing. In conclusion, 2.58 mt seems to be the slowest avisynth build among 2.57 - 2.60 in my tests.

As a preface to benchmark methodology, I personally have a script that just opens a video file and plays it back, not actual editing involved. At the very least, I can playback or encode through x64 and 32bit versions at the same speed, minus filtering. I mean, literally the same. I've been testing against 2.6 alpha 2, because it has the snazzy new resize code in full effect, minus SSE2-ized horizontal resize code. I also do not have SSE2 horizontal code implemented, mostly because I haven't come up with a "good" way of doing this.

Personally, with heavy filtering and threading my scripts, I get at least 100% performance gain, but I have to let it use 4 cores, so it definitely does not scale perfectly linearly (what does these days ;)) with the number of processors. Is the 2.58 mt build that's slowest for you the 32 bit version? My performance tests put my build (and the x64 filters to go with it) at least as fast as the 2.6 alpha 2, often a good deal faster when using filters like MVTools2 and Masktools that can take advantage of the extra registers that are available under 64bit mode.

What tests have you been running to gauge the speeds between the various builds? This will give me a better idea of where the code needs sprucing up. I'm aware the color space conversions, notably yuy2-->anything else, are pretty crappy. What else needs a good kick in the seat of the pants?

This goes for anyone and everyone, where are you guys seeing speed bottlenecks?

I haven't had time to look into the x264 weirdness, but hopefully that is something that can be taken care of swiftly, I'll look it over in my free time today. Here's the code that appears to be the source of the error, now to find out why . . . might have to wonder over to doom10 and look like a total idiot . . .


if( pic.i_pts <= largest_pts )
{
if( param->i_log_level >= X264_LOG_WARNING )
{
if( param->i_log_level >= X264_LOG_DEBUG || pts_warning_cnt < MAX_PTS_WARNING )
fprintf( stderr, "x264 [warning]: non-strictly-monotonic pts at frame %d (%"PRId64" <= %"PRId64")\n",
i_frame, output_pts, largest_pts * dts_compress_multiplier );
else if( pts_warning_cnt == MAX_PTS_WARNING )
fprintf( stderr, "x264 [warning]: too many nonmonotonic pts warnings, suppressing further ones\n" );
pts_warning_cnt++;
}
pic.i_pts = largest_pts + ticks_per_frame;
output_pts = pic.i_pts * dts_compress_multiplier;
}


EDIT: This appears to only happen when x264 cannot understand the input file format. For example, if I explicitly tell x264 my clip is interlaced and at 30000/1001 (29.97fps), no warnings are displayed. The Avisynth support in x264 was intended to support 32bit Avisynth, perhaps there's something not being read from the avisynth64 input stream?

Turbojet, no discouragement taken, you've been very helpful, and always provide constructive and intelligent feedback. What more could a community driven developer ask for? This project started as a way to keep me busy, and wouldn't have gotten very far if people didn't take interest and provide feedback.

@levi
I wholeheartedly agree with your synopsis of the caveats that threading a script brings. Hyperthreading may slow your encodes down because, in essence, hyperthreading is a hack. Interestingly enough, AMD engineers publicly called it so when Intel introduced it. It just allows two threads access to your processors execution pipeline at the same time. Let me show you the shiny block diagram of the latest Core architecture from instruction issue to retirement. End story, the goal of hyperthreading is to keep all the execution units on your processor happily chunking through data.

http://www.mediafire.com/imgbnc.php/9c79e436e44bcdcbce3e37755d95640f4g.jpg

The Core architecture (latest, starting with Nehalem and its derivatives) can schedule 5 instructions (5 is a special case with macro op fusion, without, it can schedule/retire 4 instructions per clock) per clock, so keeping all the execution units busy can be difficult for some types of code. I'm guessing that encoding programs really have no problem abusing your processor, so hyperthreading can get in the way of efficient code. A lot of the code in Avisynth and x264 is hand written assembler, and it doesn't get much more efficient than that . . . as long as it's well written assembler.

I keep a log of my fastest script & threading combinations.

Any chance of sharing some of that log information? I think it would be interesting from a development standpoint.

[osgZach]

What I mean is mostly placebo is people see multithreading avisynth and assume it's faster but in most cases it's actually slower. I don't know of a gui that makes use of it, it's not a complicated addition but it slows down all the functions they provide.

The assumption isn't necessarily wrong because almost all multithreaded programs have significant performance increase over their single threaded counterpart.

By cutting encoding time in half you must be using some very heavy filters, what exactly are you using?

I only use MT on TGMC currently, which obviously is a very heavy filter. But its one of the few I need/use. I don't typically use heavy denoisers or anything like that, and typically I will just use a default "LSFMod()" call to clean things up a little. I haven't tried seeing what effect MT or SetMTMode would have with it however.

Also worth noting, I don't run filters on x264 encodes, except something with negligent impact such as "Undot()". I do all my filtering into a HuffYV12 first. But lately I have been working from a decent DVD source.. I have other DVD's which are quite horrible and have other problems of their own.. So I won't be able to do further broad comparisons until I get around to doing them (and at that point I hope to have a Core2 quad or i5 at least).

[Jeremy Duncan]

I see your using SET's 2.5.8. It's been a while since I saw it, but, if you look at the files in the project not all of the files in the src are in there. You should add the missing files to the project and then some bugs people are reporting will be fixed. I told SEt about this, maybe he fixed it since them?

[Stephen R. Savage]

I am getting an unknown exception with LaTo's GradFun2DBmod with the latest 4/01 build at resolutions 1440x1080 and 1280x720.

Edit: This also occurs with the 3/19 build. I am currently using GradFunkMirror as a replacement, and will see if I can isolate the script lines that cause this.

Edit2: I have traced the exception to the following fragment:

grain = temp<=0 ? grain
\ : temp==100 ? mt_makediff(dither,diff.temporalsoften(1,255,chr?255:0,255,2),u=chr?3:2,v=chr?3:2)
\ : mt_makediff(dither,diff.mergeluma(diff.temporalsoften(1,255,chr?255:0,255,2),temp/100.0),u=chr?3:2,v=chr?3:2)

The portion that is in effect is:

mt_makediff(dither,diff.mergeluma(diff.temporalsoften(1,255,0,255,2),50.0/100.0),u=2,v=2)

I can reproduce the exception with:

BlankClip()
ConvertToYV12()
MergeLuma(TemporalSoften(1, 255, 0, 255, 2), 50.0 / 100.0)

One instance of this managed to crash VirtualDub64, and the backtrace is:

VirtualDub crash report -- build 32706 (release-AMD64)
--------------------------------------

Disassembly:
7feed284820: c410 les edx, fword ptr [eax]
7feed284822: 0100 add [eax], eax
7feed284824: 005dc3 add [ebp-3dh], bl
7feed284827: 660f db 0fh
7feed284829: 1f pop ds
7feed28482a: 8400 test [eax], al
7feed28482c: 0000 add [eax], al
7feed28482e: 0000 add [eax], al
7feed284830: 57 push edi
7feed284831: 48 dec eax
7feed284832: 83ec30 sub esp, 30h
7feed284835: 83792000 cmp dword ptr [ecx+20h], 00h
7feed284839: 7423 jz ed28485e
7feed28483b: 48 dec eax
7feed28483c: 8b7918 mov edi, [ecx+18h]
7feed28483f: 48 dec eax
7feed284840: 8b4f38 mov ecx, [edi+38h]
7feed284843: 48 dec eax
7feed284844: 8b01 mov eax, [ecx]
7feed284846: 4c dec esp
7feed284847: 8b5730 mov edx, [edi+30h]
7feed28484a: 4d dec ebp
7feed28484b: 63c0 arpl ax, ax
7feed28484d: 4d dec ebp
7feed28484e: 63c9 arpl cx, cx
7feed284850: 4c dec esp
7feed284851: 89542420 mov [esp+20h], edx
7feed284855: ff5018 call dword ptr [eax+18h]
7feed284858: 48 dec eax
7feed284859: 83c430 add esp, 30h
7feed28485c: 5f pop edi
7feed28485d: c3 ret
7feed28485e: e80d000000 call ed284870
7feed284863: 48 dec eax
7feed284864: 83c430 add esp, 30h
7feed284867: 5f pop edi
7feed284868: c3 ret
7feed284869: 0f db 0fh
7feed28486a: 1f pop ds
7feed28486b: 800000 add byte ptr [eax], 00h
7feed28486e: 0000 add [eax], al
7feed284870: 55 push ebp
7feed284871: 48 dec eax
7feed284872: 81ec90000000 sub esp, 00000090
7feed284878: 48 dec eax
7feed284879: 8d6c2420 lea ebp, [esp+20h]
7feed28487d: 4c dec esp
7feed28487e: 897d30 mov [ebp+30h], edi
7feed284881: 4c dec esp
7feed284882: 897538 mov [ebp+38h], esi
7feed284885: 4c dec esp
7feed284886: 896d40 mov [ebp+40h], ebp
7feed284889: 4c dec esp
7feed28488a: 896510 mov [ebp+10h], esp
7feed28488d: 48 dec eax
7feed28488e: 897d18 mov [ebp+18h], edi
7feed284891: 48 dec eax
7feed284892: 897520 mov [ebp+20h], esi
7feed284895: 48 dec eax
7feed284896: 895d28 mov [ebp+28h], ebx
7feed284899: 48 dec eax
7feed28489a: 89d6 mov esi, edx
7feed28489c: 48 dec eax
7feed28489d: 89cb mov ebx, ecx
7feed28489f: 48 dec eax
7feed2848a0: 8b4318 mov eax, [ebx+18h]
7feed2848a3: 48 dec eax
7feed2848a4: c74500feffffff mov dword ptr [ebp+00h], fffffffe
7feed2848ab: 48 dec eax
7feed2848ac: 8b4838 mov ecx, [eax+38h]
7feed2848af: 4c dec esp
7feed2848b0: 8b11 mov edx, [ecx]
7feed2848b2: 48 dec eax
7feed2848b3: 8d5508 lea edx, [ebp+08h]
7feed2848b6: 4c dec esp
7feed2848b7: 8b4830 mov ecx, [eax+30h]
7feed2848ba: 41 inc ecx
7feed2848bb: ff5208 call dword ptr [edx+08h]
7feed2848be: 90 nop <-- FAULT
7feed2848bf: 48 dec eax
7feed2848c0: 8b4508 mov eax, [ebp+08h]
7feed2848c3: 48 dec eax
7feed2848c4: 85c0 test eax, eax
7feed2848c6: 7515 jnz ed2848dd
7feed2848c8: 48 dec eax
7feed2848c9: 8b4318 mov eax, [ebx+18h]
7feed2848cc: 48 dec eax
7feed2848cd: 8b4830 mov ecx, [eax+30h]
7feed2848d0: 48 dec eax
7feed2848d1: 8b39 mov edi, [ecx]
7feed2848d3: 48 dec eax
7feed2848d4: 8d15c2cc2200 lea edx, [0022ccc2]
7feed2848da: ff5728 call dword ptr [edi+28h]
7feed2848dd: 48 dec eax
7feed2848de: 8b4318 mov eax, [ebx+18h]
7feed2848e1: 48 dec eax
7feed2848e2: 8b4838 mov ecx, [eax+38h]
7feed2848e5: 48 dec eax
7feed2848e6: 8b11 mov edx, [ecx]
7feed2848e8: ff5228 call dword ptr [edx+28h]
7feed2848eb: 48 dec eax
7feed2848ec: 8b4d08 mov ecx, [ebp+08h]
7feed2848ef: 8b5814 mov ebx, [eax+14h]
7feed2848f2: 44 inc esp
7feed2848f3: 8b4114 mov eax, [ecx+14h]
7feed2848f6: 44 inc esp
7feed2848f7: 8b611c mov esp, [ecx+1ch]
7feed2848fa: ba02000000 mov edx, 00000002
7feed2848ff: 44 inc esp
7feed284900: 894550 mov [ebp+50h], eax
7feed284903: 44 inc esp
7feed284904: 8b4118 mov eax, [ecx+18h]
7feed284907: 44 inc esp
7feed284908: 894558 mov [ebp+58h], eax
7feed28490b: 41 inc ecx
7feed28490c: 8d4003 lea eax, [eax+03h]
7feed28490f: 83e0fc and eax, 0fch
7feed284912: 894548 mov [ebp+48h], eax
7feed284915: e816e90b00 call ed343230
7feed28491a: 44 inc esp
7feed28491b: 8b4558 mov eax, [ebp+58h]
7feed28491e: 83 db 83h
7feed28491f: c0 db 0c0h

Built on Aegis on Thu Dec 24 19:58:44 2009 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7600) []

RAX = 0
RBX = 26513c0
RCX = 12ed80
RDX = fdc9d0f4
RSI = 1d84c30
RDI = 1d84c30
RBP = 12f280
R8 = 12ec68
R9 = 12f280
R10 = 0
R11 = 286
R12 = 0
R13 = 12f5d8
R14 = 12f5d0
R15 = 70800
RSP = 12ecb0
RIP = 7feed475109
EFLAGS = 00000202


Crash reason: unknown exception 0x40000015

Crash context:
An exception occurred in module 'Veedub64'.

Pointer dumps:

RBX 026513c0: ed573500 000007fe ed573480 000007fe 00000001 00000000 02625000 00000000
RCX 0012ed80: 006124b0 00000000 000000d5 00000000 002f4f70 00000000 00000000 00000000
RSI 01d84c30: efefefef efefefef efefefef efefefef efefefef efefefef efefefef efefefef
RDI 01d84c30: efefefef efefefef efefefef efefefef efefefef efefefef efefefef efefefef
RSP 0012ecb0: 0012ecd0 00000000 00000006 00000000 00000000 00000000 00000001 00000000
0012ecd0: 0012ece0 00000000 0012ed80 00000000 40000015 00000000 00000000 00000000
0012ecf0: ed2848be 000007fe 00000000 00000000 00000000 00000000 00000000 00000000
0012ed10: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
RBP 0012f280: fffffffe ffffffff 00000000 00000000 00000000 00000000 01d84c30 00000000
0012f2a0: 01d84c30 00000000 00000036 00000000 00070800 00000000 0012f5d0 00000000
0012f2c0: 0012f5d8 00000000 7753c47b 00000000 01d7f390 00000000 00000020 00000000
0012f2e0: 0027056c 00000000 02000001 00000000 0012f340 00000000 ed284504 000007fe
R8 0012ec68: 77667dba 00000000 00070800 00000000 0012f5d0 00000000 0012f5d8 00000000
R9 0012f280: fffffffe ffffffff 00000000 00000000 00000000 00000000 01d84c30 00000000
R13 0012f5d8: 00000001 00000001 00000001 00000000 40122499 00000001 00070800 00000000
R14 0012f5d0: 00070800 00000000 00000001 00000001 00000001 00000000 40122499 00000001

Thread call stack:
7feed475109: avisynth!avs_at_exit [7feed230000+f8730+14c9d9]
7feed2848be: avisynth!avs_release_video_frame [7feed230000+4dcd0+6bee]
7feed475109: avisynth!avs_at_exit [7feed230000+f8730+14c9d9]
77682075: ntdll!RtlRestoreLastWin32Error [77630000+52040+35]
775a1511: USER32!MessageBoxTimeoutA [77530000+7135c+1b5]
7754c296: USER32!GetSystemMetrics [77530000+1c110+186]
775a11d6: USER32!MessageBoxA [77530000+71188+4e]
7feed488cb5: avisynth!avs_at_exit [7feed230000+f8730+160585]
7feed47e560: avisynth!avs_at_exit [7feed230000+f8730+155e30]
7feed4767a5: avisynth!avs_at_exit [7feed230000+f8730+14e075]
7feed2848be: avisynth!avs_release_video_frame [7feed230000+4dcd0+6bee]
7754a01b: USER32!IsDialogMessageW [77530000+19e80+19b]
7753c47b: USER32!RegisterPowerSettingNotification [77530000+c328+153]
7feed284504: avisynth!avs_release_video_frame [7feed230000+4dcd0+6834]
7fefbed1381: uxtheme!00001381
7767fdf5: ntdll!KiUserCallbackDispatcher [77630000+4fdd6+1f]
7754c3df: USER32!GetSystemMetrics [77530000+1c110+2cf]
7feed284128: avisynth!avs_release_video_frame [7feed230000+4dcd0+6458]
7fef59d783a: AVIFIL32!AVIStreamRead [7fef59d0000+7810+2a]
1401085e2: AVIReadTunnelStream::Read()
7754a46a: USER32!GetSysColor [77530000+1a418+52]
7fefd6a2593: COMCTL32!DefSubclassProc [7fefd680000+222ec+2a7]
7754a54d: USER32!IsWindow [77530000+1a544+9]
14003ec2d: VDFilterFrameBuffer::AddRef()
140122ae2: VideoSourceAVI::_read()
7fefd6a24af: COMCTL32!DefSubclassProc [7fefd680000+222ec+1c3]
140043bb6: VDFilterFrameVideoSource::RunRequests()
1400a6497: VDUIFrame::DefProc()
7754c3df: USER32!GetSystemMetrics [77530000+1c110+2cf]
14008d777: VDProjectUI::GetFrameString()
140083775: VDProject::UpdateFrame()
140042d0a: VDFilterFrameRequestAllocator::Allocate()
140041f91: VDFilterFrameQueue::GetRequest()
1400a5a93: VDTimeline::TimelineToSourceFrame()
1400846ec: VDProject::DisplayFrame()
14008fbc7: VDProjectUI::UICurrentPositionUpdated()
140081915: VDProject::MoveToFrame()
14008cbdb: VDProjectUI::OnPositionNotify()
1400952fe: VDProjectUI::MainWndProc()
7fefbed1644: uxtheme!00001644
7754a2d4: USER32!IsDialogMessageW [77530000+19e80+454]
7754c296: USER32!GetSystemMetrics [77530000+1c110+186]
77546072: USER32!GetWindowLongPtrA [77530000+16050+22]
1400a70a8: VDUIFrame::StaticWndProc()
7753e034: USER32!GetQueueStatus [77530000+de20+214]
7754c3c1: USER32!GetSystemMetrics [77530000+1c110+2b1]
1400f1bba: VDPositionControlW32::WndProc()
7754c333: USER32!GetSystemMetrics [77530000+1c110+223]
1400f1f82: VDPositionControlW32::StaticWndProc()
7754a6d8: USER32!IsWindow [77530000+1a544+194]
7753fd38: USER32!SendMessageA [77530000+fcd8+60]
77547d86: USER32!GetWindowLongA [77530000+17d64+22]
1400f1b66: VDPositionControlW32::WndProc()
1400f1f82: VDPositionControlW32::StaticWndProc()
7754c3c1: USER32!GetSystemMetrics [77530000+1c110+2b1]
7767fdf5: ntdll!KiUserCallbackDispatcher [77630000+4fdd6+1f]
7754c333: USER32!GetSystemMetrics [77530000+1c110+223]
7754c60a: USER32!GetSystemMetrics [77530000+1c110+4fa]
7754bd32: USER32!TranslateMessage [77530000+1bd10+22]
14006ec2e: inMain()
14022cb31: _tmainCRTStartup()
776cd94a: ntdll!MD5Final [77630000+93950+9ffa]
7742f56d: kernel32!BaseThreadInitThunk [77410000+1f560+d]
77663281: ntdll!RtlUserThreadStart [77630000+33260+21]

-- End of report

[squid_80]

EDIT: This appears to only happen when x264 cannot understand the input file format. For example, if I explicitly tell x264 my clip is interlaced and at 30000/1001 (29.97fps), no warnings are displayed. The Avisynth support in x264 was intended to support 32bit Avisynth, perhaps there's something not being read from the avisynth64 input stream?


If x264 is including avisynth.h, it will need the fixes to the AVSValue =operator to be 64-bit compatible.

[kemuri-_9]

If x264 is including avisynth.h, it will need the fixes to the AVSValue =operator to be 64-bit compatible.

x264 uses avisynth_c.h for the C interface, not avisynth.h for the standard C++ one.
and we do include this in x264's repository so as to not have users compiling for windows require it.