HELP! 3 days of crashes w avisynth/lagarith/fft3d

[fbs]

For 3 days Im trying to denoise a 2h long clip vhs 720x480 captured with lagarith (latest version), using vdub+avisynth and this simple script:


setmemorymax(700)
setmtmode(5, 7)

AviSource("d:\ds000.avi", true)
setmtmode(2)

AssumeBFF()
crop(16, 1, -4, -7)

converttoyv12()
fft3dfilter(interlaced=true, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=0, plane=4, wintype=2,px=1,py=2,pframe=215546)


Everytime, after 90% done it crashes.. sometimes "accusing" lagarith.dll as the culprit.. I've had the same issues with MCTemporalDenoise and that's why I changed to fft3d, believing it'd be more stable..

Im using set's 2.6MT avisynth and had problems with 2.5.8 too as far as I remember..

Is there any known bug with lagarith? ds000.avi is compressed with it (yuy2) and I'm outputting compressed with it too (RGB)..

Or is this "clip" too long? 2h... :/ RAM usage for vdub was 1.0gb (read somewhere there's a 1.5gb limit or something)

vdub says almost always something like this:
Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'lagarith'...

...reading address 05A3A000.


ps.: my pc is stable.. run prime95.exe for 6h without crashes.. no BSODs ever!!
ps2.: win7 ult x64 4gb phenom ii x3 720be

[the_weirdo]

You can try Ut Video (http://www.videohelp.com/tools/Ut-Video-Codec-Suite), it's much faster and stable than Lagarith.

[karesch]

I had similar issues with FFT3dGPU and Lagarith, where it always crashed at a particular frame(s). After checking what was on that frame, it turned out that there were frames with camcorder scene changes, where the current and the next scenes are mixed, either on the top half-screen the new scene, at the bottom half-screen the current one, or frames with the new scene where some of the macroblocks are from the old frame or fuzzy.

This usually happens with camcorder sources. If your source is from a camcorder, you might check the frames where the crash occurs, and the frames have problems like the ones below, you might want to delete them and restart the script.

Mixed frame (half/half):
http://img225.imageshack.us/img225/9410/halfa.th.jpg (http://imageshack.us/photo/my-images/225/halfa.jpg/)

Uploaded with ImageShack.us (http://imageshack.us)

Mixed macroblocks:
http://img207.imageshack.us/img207/8352/mixedmacroblocks.th.jpg (http://img207.imageshack.us/i/mixedmacroblocks.jpg/)

[fbs]

setmtmode(5, 7) is the cause. Decrease it to setmtmode(5, 3) or (5, 2). See my x64 FAQ for details (the same for x86). Also decrease a thread count in FFT3DFilter.

are you sure about it? I won't get 100% cpu usage with 3 threads only... :(

[fbs]

I had similar issues with FFT3dGPU and Lagarith, where it always crashed at a particular frame(s). After checking what was on that frame, it turned out that there were frames with camcorder scene changes, where the current and the next scenes are mixed, either on the top half-screen the new scene, at the bottom half-screen the current one, or frames with the new scene where some of the macroblocks are from the old frame or fuzzy.

This usually happens with camcorder sources. If your source is from a camcorder, you might check the frames where the crash occurs, and the frames have problems like the ones below, you might want to delete them and restart the script.


It won't crash always on the same frame.. I thought of a "bad frame" too.. but it has crashed on ~50%, but most of the time it's from 90% onwards just to make me mad. That's why I thought maybe the video is too long for avisynth... ?

[TheFluff]

are you sure about it? I won't get 100% cpu usage with 3 threads only... :(

Try and calculate your average CPU usage over these three days you've been trying to encode this clip and see what kind of figures you end up with.

(Moral of this story: a slow process that finishes its work is infinitely much faster than a fast one that always crashes before it is finished, because the fast one never completes its work at all.)

[fbs]

yes I know. but I'm not sure its the number of threads that's crashing.. qtgmc recommends you to raise it till it uses 100% cpu..

do you know if MCTemporalDenoise CAN be multithreaded at all?

[fbs]

right now I'm using 2 different .avss. One for the first half of the video and another for the second half... each one using 3 threads only.. two vdub instances running..

Someone must track down these crashes.. I feel like using windows 98 again :(

[TheFluff]

right now I'm using 2 different .avss. One for the first half of the video and another for the second half... each one using 3 threads only.. two vdub instances running..

Someone must track down these crashes.. I feel like using windows 98 again :(

Welcome to Avisynth-MT, taking you straight back to the 90's.

[-Vit-]

Make sure you are using the latest of SEt's 2.6MT builds. Get the modded fft3dfilter from the QTGMC modded plugins collection - it has SEt's fixes in it also.
Yes, increase threads to get about 100% (or just below), but decrease if you have stability issues.

fft3dfilter has the ncpu setting. Never tried it but as that's the only significant filter in your script, could you perhaps drop the SetMTMode and try ncpu instead?

[fbs]

Thanx vit.. I was using MCTemporalDenoise, then I changed to fft3d trying to get stable but didn't work either..
Now I'll try MCTemporalDenoise again, this time with all your MTfixed dll pack (including fft3dfilter) and only 3 threads +_+


setmemorymax(700)
setmtmode(5, 3)

AviSource("d:\ds000.avi", true)
setmtmode(2)

AssumeBFF()
crop(16, 1, -4, -7)
converttoyv12()
MCTemporalDenoise(settings="low",enhance=true,stabilize=true,useTTmpSm=true,interlaced=true,sharp=false,protect=true,chroma=true,edgeclean=true,truemotion=true,bt=5,sigma=2)
#fft3dfilter(interlaced=true, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=0, plane=4, wintype=2,px=1,py=2,pframe=215546)

trim(0,107800)


oh and ncpu is almost useless :/

[chainring]

yes I know. but I'm not sure its the number of threads that's crashing.. qtgmc recommends you to raise it till it uses 100% cpu..

do you know if MCTemporalDenoise CAN be multithreaded at all?I just messed around with MCTD usage through MeGUI (not a problem, I know) and here's what I found to work.

I have the following:
i7 2600k
4 GB RAM
Win7 Ult x64

# Set DAR in encoder to 12 : 5. The following line is for automatic signalling
global MeGUI_darx = 12
global MeGUI_dary = 5
SetMemoryMax(1024)
SetMTMode(5,4)
LoadPlugin("C:\Apps\MeGUI\tools\ffms\ffms2.dll")
FFVideoSource("path to source\source.xxx", threads=1).AssumeFPS(24000,1001)
SetMTMode(2)
Spline36Resize(1280,528) # Spline36 (Neutral)
SetMTMode(4)
MCTemporalDenoise(settings="low")

[fbs]

hum.. well, it worked.. but I don't know exactly what made it work.. because I changed everything.. :P
If anyone have the same issues, here's what I did to stop crashing:
SEt's avisynth 2.6MT
all plugins with custom fixes of QTGMC pack
only 3 threads instead of 7
half the video length (trim)
and UT video codec instead of lagarith
I encoded simultaneously with two vdubs instances running.. each one encoding half length of the video.. so of course it was waaay faster than only one at a time with only 3 threads (40% cpu usage only)

but now I have another problem.. the outputed .avi is crashing Vegas 10 x64.. don't tell me UTVideo can't be opened by vegas, please.... :(


Problem signature:
Problem Event Name: APPCRASH
Application Name: vegas100.exe
Application Version: 10.0.0.738
Application Timestamp: 4e0a4fd7
Fault Module Name: utv_core.dll
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 4e8c5636
Exception Code: c0000005
Exception Offset: 000000000001ec9c
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1046

[fbs]

You can try Ut Video (http://www.videohelp.com/tools/Ut-Video-Codec-Suite), it's much faster and stable than Lagarith.

grrr tried vegas x86 and x64.. both crash when I try to open the .avi with utvideo codec

[karesch]

grrr tried vegas x86 and x64.. both crash when I try to open the .avi with utvideo codec

well, I hate telling bad news, but utvideo does not work with my Vegas 10.0 64bit either (on Windows 7), it only seemed to work flawlessly with VirtualDub and x264, it behaves strangely in MPC and in WMP

[fbs]

Well, as I couldn't open UTVideo with my favorite NLE (Sony Vegas x86 AND x64), I tried to encode again with lagarith. This time using the above mentioned changes (only 3 threads, MT-fixed plugins etc..)
And for my suprise... CRASH again, in both vdub instances (first and second half of the source video) one in 84% and other in 13% (this doesn't matter.. everytime is in a different %).. memory usage for both instances was near 512mb and this is far for any process size barrier. So, it seems lagarith 1.3.26 is really f#$@# up! The next step in my crash-journey is huffyuv... hehe

VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
0334ede0: 83ec18 sub esp, 18h
0334ede3: 53 push ebx
0334ede4: 55 push ebp
0334ede5: 8b6c2424 mov ebp, [esp+24h]
0334ede9: 56 push esi
0334edea: 33d2 xor edx, edx
0334edec: 57 push edi
0334eded: b880000000 mov eax, 00000080
0334edf2: c7442410010000 mov dword ptr [esp+10h], 00000001
00
0334edfa: c7442414020000 mov dword ptr [esp+14h], 00000002
00
0334ee02: c7442418030000 mov dword ptr [esp+18h], 00000003
00
0334ee0a: c744241c050000 mov dword ptr [esp+1ch], 00000005
00
0334ee12: c7442420080000 mov dword ptr [esp+20h], 00000008
00
0334ee1a: c74424240d0000 mov dword ptr [esp+24h], 0000000d
00
0334ee22: 8954242c mov [esp+2ch], edx
0334ee26: 33c9 xor ecx, ecx
0334ee28: 33f6 xor esi, esi
0334ee2a: 33ff xor edi, edi
0334ee2c: 8d5c2410 lea ebx, [esp+10h]
0334ee30: 85f6 test esi, esi
0334ee32: 7404 jz 0334ee38
0334ee34: 85c9 test ecx, ecx
0334ee36: 7521 jnz 0334ee59
0334ee38: 8bf1 mov esi, ecx
0334ee3a: 0fb60c2a movzx ecx, byte ptr [edx+ebp]
0334ee3e: 23c8 and ecx, eax
0334ee40: d1e8 shr eax, 1
0334ee42: 7506 jnz 0334ee4a
0334ee44: 42 inc edx
0334ee45: b880000000 mov eax, 00000080
0334ee4a: 85c9 test ecx, ecx
0334ee4c: 7406 jz 0334ee54
0334ee4e: 85f6 test esi, esi
0334ee50: 7502 jnz 0334ee54
0334ee52: 033b add edi, [ebx]
0334ee54: 83c304 add ebx, 04h
0334ee57: ebd7 jmp 0334ee30
0334ee59: 4f dec edi
0334ee5a: be01000000 mov esi, 00000001
0334ee5f: 741a jz 0334ee7b
0334ee61: 0fb60c2a movzx ecx, byte ptr [edx+ebp] <-- FAULT
0334ee65: 23c8 and ecx, eax
0334ee67: d1e8 shr eax, 1
0334ee69: 7506 jnz 0334ee71
0334ee6b: 42 inc edx
0334ee6c: b880000000 mov eax, 00000080
0334ee71: 03f6 add esi, esi
0334ee73: 85c9 test ecx, ecx
0334ee75: 7401 jz 0334ee78
0334ee77: 46 inc esi
0334ee78: 4f dec edi
0334ee79: 75e6 jnz 0334ee61
0334ee7b: 8b4c2430 mov ecx, [esp+30h]
0334ee7f: 8b7c242c mov edi, [esp+2ch]
0334ee83: 4e dec esi
0334ee84: 8934b9 mov [ecx+edi*4], esi
0334ee87: 7573 jnz 0334eefc
0334ee89: 33c9 xor ecx, ecx
0334ee8b: 33f6 xor esi, esi
0334ee8d: 33ff xor edi, edi
0334ee8f: 8d5c2410 lea ebx, [esp+10h]
0334ee93: 85f6 test esi, esi
0334ee95: 7404 jz 0334ee9b
0334ee97: 85c9 test ecx, ecx
0334ee99: 7521 jnz 0334eebc
0334ee9b: 8bf1 mov esi, ecx
0334ee9d: 0fb60c2a movzx ecx, byte ptr [edx+ebp]
0334eea1: 23c8 and ecx, eax
0334eea3: d1e8 shr eax, 1
0334eea5: 7506 jnz 0334eead
0334eea7: 42 inc edx
0334eea8: b880000000 mov eax, 00000080
0334eead: 85c9 test ecx, ecx
0334eeaf: 7406 jz 0334eeb7
0334eeb1: 85f6 test esi, esi
0334eeb3: 7502 jnz 0334eeb7
0334eeb5: 033b add edi, [ebx]
0334eeb7: 83c304 add ebx, 04h
0334eeba: ebd7 jmp 0334ee93
0334eebc: 4f dec edi
0334eebd: be01000000 mov esi, 00000001
0334eec2: 741a jz 0334eede
0334eec4: 0fb60c2a movzx ecx, byte ptr [edx+ebp]
0334eec8: 23c8 and ecx, eax
0334eeca: d1e8 shr eax, 1
0334eecc: 7506 jnz 0334eed4
0334eece: 42 inc edx
0334eecf: b880000000 mov eax, 00000080
0334eed4: 03f6 add esi, esi
0334eed6: 85c9 test ecx, ecx
0334eed8: 7401 jz 0334eedb
0334eeda: 46 inc esi
0334eedb: 4f dec edi
0334eedc: 75e6 jnz 0334eec4
0334eede: 8b db 8bh
0334eedf: 4c dec esp

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 00000080
EBX = 0653f148
ECX = 00000000
EDX = 000751e6
EBP = 0603be1a
ESI = 00f00200
EDI = 084106c0
ESP = 0653f110
EIP = 0334ee61
EFLAGS = 00210206
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'lagarith'...

...reading address 060B1000.

Pointer dumps:

EBX 0653f148: 0603be19 068e0030 00054600 03353ea9 05465844 0603be1a 05465844 a303b941
ESI 00f00200: 0002c018 00000000 0000000e 00000000 0002c037 00000000 ffffffff ffffffff
EDI 084106c0: ffffffff ffffffff ffedd565 fff0da72 ffccb67d ff87681e 22000000 22000000
ESP 0653f110: 00000000 0603be19 05465844 00054600 00000001 00000002 00000003 00000005
0653f130: 00000008 0000000d 03353d87 0000000a 05465848 05465844 0603be19 068e0030
0653f150: 00054600 03353ea9 05465844 0603be1a 05465844 a303b941 e703b93e 0334fe27
0653f170: 068e0030 00054600 068e0030 05465810 0695e930 06934630 00044fc8 0603be19
EBP 0603be1a: 2ffafffc 2f48ffff 2e090002 2ddbfffa 2f1cfffe 2e820003 2bcb0000 2711ffff
0603be3a: 237efffe 2197fff9 2134fffc 21b0ffff 21c0ffff 21060002 1f5f0003 1c41fffe
0603be5a: 1974ffff 177afffd 15a7fff8 143effff 116dfff9 0e4ffffc 0ce4fffe 0b27fffb
0603be7a: 09b3fff9 076c0003 03fdfffb ff88fffd fc4efffc fcfffff9 009d0000 02510002

Thread call stack:
0334ee61: lagarith!0001ee61
03353d87: lagarith!00023d87
03353ea9: lagarith!00023ea9
0334fe27: lagarith!0001fe27
03350289: lagarith!00020289
03350652: lagarith!00020652
03354132: lagarith!DriverProc [03330000+23fe0+152]
72ef1759: MSVFW32!ICSendMessage [72ef0000+1728+31]
72ef4e27: MSVFW32!ICDecompress [72ef0000+4dea+3d]
030cea4d: AviSynth!DllCanUnloadNow [02fd0000+12200+ec84d]
02fd588f: AviSynth!0000588f
77a43ca3: ntdll!RtlImageNtHeader [77a10000+33164+b3f]
030d02ea: AviSynth!DllCanUnloadNow [02fd0000+12200+ee0ea]
02fe1e7c: AviSynth!avs_delete_script_environment [02fd0000+b4e0+699c]
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
77a3e0d2: ntdll!RtlAllocateHeap [77a10000+2e026+ac]
77a3f9f9: ntdll!RtlAnsiCharToUnicodeChar [77a10000+2f91a+df]
02fd14d6: AviSynth!000014d6
02fe1301: AviSynth!avs_delete_script_environment [02fd0000+b4e0+5e21]
02fdf699: AviSynth!avs_delete_script_environment [02fd0000+b4e0+41b9]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
02fd7587: AviSynth!00007587
030aea04: AviSynth!DllCanUnloadNow [02fd0000+12200+cc804]
77a43ca3: ntdll!RtlImageNtHeader [77a10000+33164+b3f]
77a43cce: ntdll!RtlImageNtHeader [77a10000+33164+b6a]
77a40058: ntdll!LdrGetDllHandleEx [77a10000+2fd18+340]
77a3fd0f: ntdll!LdrGetDllHandle [77a10000+2fcf7+18]
71c23eb8: MSVCR90!??2@YAPAXI@Z [71bc0000+63e99+1f]
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
77a2f8c1: ntdll!NtWaitForSingleObject [77a10000+1f8ac+15]
77a48dd4: ntdll!RtlIntegerToUnicodeString [77a10000+38aad+327]
02fd7b3d: AviSynth!00007b3d
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
02fd81ba: AviSynth!000081ba
02fe1301: AviSynth!avs_delete_script_environment [02fd0000+b4e0+5e21]
02fdf699: AviSynth!avs_delete_script_environment [02fd0000+b4e0+41b9]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
030e00f4: AviSynth!DllCanUnloadNow [02fd0000+12200+fdef4]
02fe1301: AviSynth!avs_delete_script_environment [02fd0000+b4e0+5e21]
02fd7872: AviSynth!00007872
02fdf699: AviSynth!avs_delete_script_environment [02fd0000+b4e0+41b9]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
72cf4277: FFT3DFilter!00004277
062e8f04: fftw3!fftwf_execute_dft_r2c [06240000+a8ec0+44]
72cf8600: FFT3DFilter!00008600
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
02fd6a99: AviSynth!00006a99
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
77a3e0d2: ntdll!RtlAllocateHeap [77a10000+2e026+ac]
72cf9a6c: FFT3DFilter!00009a6c
02fe1301: AviSynth!avs_delete_script_environment [02fd0000+b4e0+5e21]
71c23eb8: MSVCR90!??2@YAPAXI@Z [71bc0000+63e99+1f]
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
02fdf835: AviSynth!avs_delete_script_environment [02fd0000+b4e0+4355]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
77a3e36c: ntdll!RtlInitUnicodeString [77a10000+2e208+164]
77a3e0d2: ntdll!RtlAllocateHeap [77a10000+2e026+ac]
03027fcc: AviSynth!DllCanUnloadNow [02fd0000+12200+45dcc]
02fe1301: AviSynth!avs_delete_script_environment [02fd0000+b4e0+5e21]
71c23eb8: MSVCR90!??2@YAPAXI@Z [71bc0000+63e99+1f]
02fdf835: AviSynth!avs_delete_script_environment [02fd0000+b4e0+4355]
02fdf417: AviSynth!avs_delete_script_environment [02fd0000+b4e0+3f37]
76dd0ac4: KERNELBASE!WaitForSingleObjectEx [76dc0000+109f9+cb]
75e61194: kernel32!WaitForSingleObjectEx [75e50000+11151+43]
02fe71b9: AviSynth!DllCanUnloadNow [02fd0000+12200+4fb9]
75e6339a: kernel32!BaseThreadInitThunk [75e50000+13388+12]
77a49ed2: ntdll!RtlInitializeExceptionChain [77a10000+39e6f+63]
77a49ea5: ntdll!RtlInitializeExceptionChain [77a10000+39e6f+36]

-- End of report

[Bloax]

Why don't you re-encode the finished UTVideo one with Lagarith?
Because nothing should prevent VirtualDub from doing that. (Since only Vegas has UTVideo problems.)

[Mounir]

use the gpu version, no Mt this way and no bug...