Report :- "Evaluate: Unrecognized exception!"

[IanB]

Well after many torturous hours with the debugger and a disassembler I think I have finally found the cause of the infamous "Evaluate: Unrecognized exception!" in release versions of AviSynth. It appears to be an optimiser bug in the compiler. In the generated Structured Exception Handling code the compiler forgets to initialize a stack data location on entry. When no exceptions are thrown it doesn't matter, the main line code doesn't reference this local variable (probably why the optimizer omitted the code). However when processing an exception, the SEH code references the uninitialized data from the stack and misbehaves in some fashion. If it causes an access violation you get "Evaluate: Unrecognized exception!" or if you are really lucky it quietly scribbles randomly in memory and leaves a nice little bomb waiting to detonate.

How to fix it, and be sure, well that's a problem. Doing almost anything makes the problem hide. Add some debugging code, change the order or size of variables, shuffle the code. It seems debug version code is immune, so I have been testing the various compiler optimiser switches /O[awgipstyx] and disabling "Frame-Pointer Omission" with /Oy- seems to do the trick but as the generated code is so drasticly different, it just may be hiding the issue again.

Below is a sample of some dud code clipped from an ASM compiler output listing. This is how I now classify and identify the problem.

This first bit is the entry sequence, it is normal code generated into the "_TEXT" code segment. Note the stack offsets for parameters and local variables declared at the top. It is sparse, unreferenced elements are not listed, particularly note there is no -16 entry. The entry code is typical, it sets EBP as a frame pointer, chains itself onto the exception list and bumps the stack pointer to make room for the local variables and initialize them (well some at least)._TEXT SEGMENT
_clip$ = 12
_target_width$ = 16
_target_height$ = 20
_args$ = 24
_f$ = 28
_env$ = 32
___$ReturnUdt$ = 8
_vi$ = -56
_subrange_left$ = -32
_subrange_top$ = -24
_subrange_width$ = -48
_subrange_height$ = -40
_result$ = -64
_V$ = -65
$T120214 = -56
$T120215 = -56
$T120216 = -56
$T120217 = -56
$T120218 = -56
$T120219 = -56
$T120220 = -56
$T120221 = -56
$T120233 = -52
$T120348 = -40
$T120386 = -40
__$EHRec$ = -12
?CreateResize@@YA?AVPClip@@V1@HHPBVAVSValue@
@PAVResamplingFunction@@PAVIScriptEnvironment@
@@Z PROC NEAR ; CreateResize, COMDAT

; 1602 : {

push ebp
mov ebp, esp
and esp, -8 ; fffffff8H
push -1
push $L120656
mov eax, DWORD PTR fs:__except_list
push eax
mov DWORD PTR fs:__except_list, esp
sub esp, 56 ; 00000038H
push ebx
push esi
push edi
mov DWORD PTR $T120233[esp+80], 0
....This next bit is exception handling code, it is in the special "text$x" code segment.

For each object created on the stack there is an entry that fetches the appropriate "this" address into the ECX register and then jumps to its destructor. Each is called in turn from ___CxxFrameHandler as it parses the exception data table.

Note the two references to [ebp-16], the location that was never initialized this causes the destructor to do its thing on random memory, PClip's like to decrement words, just how evil is that. :devil:
text$x SEGMENT
$L120222:
mov ebp, DWORD PTR [ebp-16]
lea ecx, DWORD PTR _clip$[ebp-4]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120223:
lea ecx, DWORD PTR _result$[ebp]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120225:
lea ecx, DWORD PTR $T120215[ebp]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120227:
lea ecx, DWORD PTR $T120217[ebp]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120229:
lea ecx, DWORD PTR $T120219[ebp]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120231:
lea ecx, DWORD PTR $T120221[ebp]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120232:
mov ebp, DWORD PTR [ebp-16]
mov eax, DWORD PTR $T120233[ebp]
and eax, 1
test eax, eax
je $L120234
mov ecx, DWORD PTR ___$ReturnUdt$[ebp-4]
jmp ??1PClip@@QAE@XZ ; PClip::~PClip
$L120234:
ret 0
$L120656:
mov eax, OFFSET FLAT:$T120615
jmp ___CxxFrameHandler
text$x ENDS
This is the exception data table, it is in the special "xdata$x" data segment. It starts at $T120615 which has a magic number. (Why is May 20, 1993 special) :D It points to 7 entries at $T120664.
xdata$x SEGMENT
$T120664 DD 0ffffffffH
DD FLAT:$L120232
DD 00H
DD FLAT:$L120222
DD 01H
DD FLAT:$L120223
DD 02H
DD FLAT:$L120225
DD 02H
DD FLAT:$L120227
DD 02H
DD FLAT:$L120229
DD 02H
DD FLAT:$L120231
$T120615 DD 019930520H
DD 07H
DD FLAT:$T120664
DD 2 DUP(00H)
DD 2 DUP(00H)
xdata$x ENDSWell where to from here.

Do we recommend /Oy- on all avisynth and filter builds?

How can we be sure /Oy- actually fixes the problem not just hides it?

IanB :confused:

[stickboy]

Which compiler are you using? VC6 or VC7?

[IanB]

@stickboy,

VC6 -- Compiler version 12.00.8804

IanB

[sh0dan]

Great (and probably exhausting) work!

I think the best way to know is to get a version "out-there", but it's your call. If we should do a last beta, could you upload a avisynth.dsp with your suggestion for custom optimizations.

C-optimizations are not _that_ critical, since most speed-critical functions are MMX implemented.

[IanB]

@Sh0dan,

More frustrating than exhausting, but yes:D

The main contribution of /Oy+ is it frees the EBP register for general use. But it doesn't achieve much when /GX, exception handling, is enabled as it forces formal frame management, which, catch 22, needs the EBP reg. ;)

What worries me most is /Oy- may be just shuffling the code and moving the bug to another day, but as you say lets get a version "out-there" and tested.

I have just checked in revision 1.19 of avisynth.dsp, which includes an /Oy- cpp option for the release build. We also need to encourage filter authors to try this option before reporting a problem. I am very surprised this bug hasn't bitten before, perhaps we have just been lucky so far or a little unluck now.

RCS file: /cvsroot/avisynth2/avisynth/src/avisynth.dsp,v
retrieving revision 1.18
retrieving revision 1.19
diff -r1.18 -r1.19
46c46
< # ADD CPP /nologo /G6 /MD /W3 /GX /Zd /O2 /Ob2 /D "NDEBUG" /D ...
---
> # ADD CPP /nologo /G6 /MD /W3 /GX /Zd /O2 /Oy- /Ob2 /D "NDEBUG" /D ...IanB

[sh0dan]

Brings back memories when I spent a day hunting a compiler-error when I optimized decomb.

I hope this removes the very annoying bombing out in vdub, when an error occurs.

Anyway. I think we should make this an RC1 - It seems like it is time for a version increment considering the changelist. :)

[IanB]

Avisynth 2.55 RC1 It has a nice ring to it.

IanB ;)

[asarian]

Avisynth 2.55 RC1 It has a nice ring to it.

IanB ;)
IanB,

I'm getting an "Evaluate: Unrecognized Exception" on using (http://forum.doom9.org/showthread.php?p=1406617#post1406617) the avisynth SUPTitle plugin. I have the latest AviSynth, version 2.5.8, installed. I even grabbed the latest replacement DirectShowSource.dll for Avisynth 2.5.8 (didn't even know it existed). This doesn't appear to be the sort of error which allows itself to be traced easily. Is there any way I could debug this further? (without needing your whole assembler environment).

Thanks.

[IanB]

"Evaluate: Unrecognized exception!" is now the last ditch exception handler, it is thrown when a system exception is caught and the saved exception code is 0, logically it should never happen. An uncompromised system will never throw exception 0.

In some of SEt's builds he removed the following line of codemov FS:[0],esp // Install new EXCEPTION_REGISTRATIONwhich of course deactivates the code that collects and saves the exception code so it can be displayed, hence you can only ever get the generic report. If you are running a build derived from SEt's modifications report the problem to that builder.

If you are running the official Dec 22, 2008 [08:46:51], 2.5.8.5, build of Avisynth.dll and you are getting this error then you are looking at some pretty serious memory scribbling to force execution through this "impossible" code path.

With memory scribbling errors and without a full development environment your best option is to try other versions of Avisynth.dll in the hope that the scribble writes over something else so you can still get a sane error report for the real problem. Try 2.5.7 and 2.6.0alpha, the code alignment may just be different enough to get a sane error report.

[asarian]

If you are running the official Dec 22, 2008 [08:46:51], 2.5.8.5, build of Avisynth.dll and you are getting this error then you are looking at some pretty serious memory scribbling to force execution through this "impossible" code path.

With memory scribbling errors and without a full development environment your best option is to try other versions of Avisynth.dll in the hope that the scribble writes over something else so you can still get a sane error report for the real problem. Try 2.5.7 and 2.6.0alpha, the code alignment may just be different enough to get a sane error report.
Thanks for replying, IanB. Yes, I was running the official Dec 22, 2008 release (just checked). I also tried the latest Avisynth 2.5.8 MT tonight, but then the Media Player Classic (associated with my avs files, for playing) doesn't even show an error any more, but just collapses into oblivion after 1 or so seconds of blackness.

Looks like I have to give up on the SupTitle plugin. :(

[IanB]

Avisynth 2.5.8 MT is derived from SEt's modified builds, so you were wasting your time, the system exception handling is missing from those builds. As I said try 2.5.7 and 2.6.0alpha their code alignment may just be different enough to get a sane error report.

[asarian]

Avisynth 2.5.8 MT is derived from SEt's modified builds, so you were wasting your time, the system exception handling is missing from those builds. As I said try 2.5.7 and 2.6.0alpha their code alignment may just be different enough to get a sane error report.
Ah yes, the missing system exception handling in the MT build would explain a few things. :)

Seems the cause of the crash was a particular .SUP file that SupTitle tripped badly over. I haven't yet figured out what precisely goes wrong (I'll hope to do that with 2.5.7); but at least I can avoid the crash now.