Garbage in "AVSI" error reporting

[Ebobtron]

Started working with functions in “avsi” files. Using 2.57 Build: Dec31, 2006.

Error reporting can have garbage in place of avsi file name.
test.avsi
1
2 function testfunction(int a, int b)
3 {
4 return String(a / b)
5 }

The above used by the following, results in an error.

crashtextdummy.avs
1
2
3
4
5
6 clip = BlankClip(length=300, width=720, height=480, pixel_type="RGB32", \
7 fps=30000, fps_denominator=1001, audio_rate=48000, stereo=TRUE, \
8 sixteen_bit=TRUE)
9
10 clip.Subtitle(testfunction(18, 0), size=48)
http://members.aol.com/ebobtron/err_1.jpg
Evaluate: divison by zero
(????, line 4)
(C:\Documents and Settings\HP_Administrator\My Documents\crashtextdummy.avs)

The "avsi" file name is corrupt.


:)
and, of course, thank you

[IanB]

It's a limitation!

.AVSI's are parsed into memory at ScriptEnvironment creation time, there is nowhere currently to store the file name, so errors just get "????".

Workaround don't use .AVSI's, explicitly import additional funct libraries.

[Mug Funky]

is it a problem if all your avsi's work?

because i've got sooo many customized functions that explicitly importing them is a bit of a problem - my avsi's are half a mile long.

[IanB]

No, as long as they do work.

Just be aware, nasty things can happen when they don't parse correctly.

[Ebobtron]

@IanB

Ok, that works for me; I can import the function scripts needed for the transition or effect desired. Functions supported by FilmCutter can be in FilmCutter’s folder, much easier to manage.

Once more, thank you.

I have another issue uncovered by the aforementioned avsi errors, maybe one of those nasty things. Please may I beg more of your time?
I introduced a similar error into anslide.avsi (http://members.aol.com/ebobtron/anslide.zip) during a house cleaning which left the errant “:”, and I began to panic. AviSynth refused to work with any program. MediaPlayerClassic began a perpetual open, leaving itself in memory when exit was called (FilmCutter‘s directshow viewer / player locked up too, GraphEdit displayed an incoherent error), VirtualDub claimed an AviSynth open failure and VirtualDubMod terminated abruptly as did FilmCutter when reopened, both causing windows to request communication with mother. Kind of scary, for a moment.

Backtracking uncovered the trouble pretty fast. Doesn’t seem to matter where in the script the “:” is.

I don’t mean to spend more time on something you have provided a solution for, kind of that hymnal page thing again.

VirtualDubMod and FilmCutter both crash because they both interface with AviSynth in a similar way at startup.
using namespace std;
void getFunctionStrings(void)
{
avisynth = new GetLib("avisynth.dll",hwnd);
if(avisynth->error)
return;

avisynth->loadFunction((void**)&CreateScriptEnvironment,"CreateScriptEnvironment");

env = CreateScriptEnvironment(2);

env = CreateScriptEnvironment(2); crashes

try{ env = CreateScriptEnvironment(2); }
catch(...)
{
MessageBox(0,"error","AviSynth",0);
return;
}

does not crash Ok, I is a little (very little) smarter than when I originally wrote this, I next try this.try
{
env = CreateScriptEnvironment(2);
}
catch(AvisynthError err)
{
MessageBox(hwnd[0],err.msg,"CreateScriptEnvironment",0);
return;
}The above works also and gives details.
script error: syntax error
(AnSide.avsi, line 37, column 1)

This solves the problem, I was happy. I contemplate that “avsi” errors are covered, what about dll / plug-in errors? I, in an attempt to cause an error I renamed a copy of a plug-in, plug-in still works, "renamed_zoom()" and "zoom()", didn’t know it was that easy to have multiple versions of a plug-in. I leave the renamed function in the plug-in folder. I kiss my granddaughter, kick my dogs, curse my wife and return to this later. :D its a joke Fizick.

If the error is introduced into the function.avsi during a FilmCutter session the chance of lock up when attempting to preview or play the script can now be dealt with. I begin coding and testing and I put the erroneous “:” back in line 37 of anslide.avsi.

No error report, just crash. Crash on open, crash, crash, crash.

try{ env = CreateScriptEnvironment(2); }
catch(AvisynthError err)
{
MessageBox(hwnd[0],err.msg,"CreateScriptEnvironment",0);
return;
}

crash
try{ env = CreateScriptEnvironment(2); }
catch(AvisynthError err)
{
MessageBox(hwnd[0],”Unknow error”, "CreateScriptEnvironment ",0);
return;
}

No Crash
?????? Remove renamed.dll from plug-in folder and crash is replaced by error report.

Any attempt I make to access “const char * const msg” is met with a crash when
“MessageBox(hwnd[0],err.msg,"CreateScriptEnvironment",0);” is crashing.

When it is working this works too.
if(err.msg)
MessageBox(hwnd[0],”Unknow error”, "CreateScriptEnvironment ",0);
When not working this crashes also.

To read that back, syntax error in function.avsi throws Script Environment error. Add renamed.dll to the plugin folder and “err.msg” crashes. Or just use this little function alone to get err.msg to crash the progam.


function testfunction(int a, int b){
return String(a / b) :
}

This test.avsi error with or without renamed.dll crashes when using err.msg and does not without it.

I can be happy with
catch(…)
{MessageBox(0,”can not create script environment”,“AviSynth error”,0)}

I don’t know weather I am reporting an error or asking for advice on how to use err.msg in the catch statement.

I can image that this is a memory issue, that will not duplicate exactly.

Thanks again. Hope this isn’t too boring for you, as it has me yawning.


I am done, for now.

[IanB]

Your use with the try/catch(AvisynthError err) is theoretically correct. I guess you need to aggressively validate the err struct before you trust it.

Without checking deaply into the code, my best guess is we need another SaveString somewhere, i.e. what err.msg points to is getting released during the stack crawlout.

Do an extra try { whatever } catch (...) { oops! } around using err.msg from the code above.

[Ebobtron]

Thanks again for your time Ian. I do not expect you or anyone here to send me back to school, the learning is my responsibility. Therefore, I am truly grateful. IanB you wrote:
I guess you need to aggressively validate the err struct before you trust it.

I have no idea what you mean by that. I like aggressive, I would happily be that if I knew how. :)

stack crawlout... You have to love the words, like "thunk".

IanB you wrote:
Do an extra try { whatever } catch (...) { oops! } around using err.msg from the code above.

Again, I am not sure what you mean, unless ....

try
{
try
{
env = CreateScriptEnvironment(2);
}
catch(AvisynthError err)
{
MessageBox(hwnd[0],err.msg,"AviSynth Script Environment Error",0);
return;
}
}
catch(...)
{
MessageBox(hwnd[0],
"AviSynth error unknow, avsFilmCutter could not create a script \n\n"
"environment. This means your AviSynth installation is screwed.\n\n"
"Go ahead, click Ok, I dare you. ","AviSynth Crashed",0);
return;
}
If this is what you have in mind, it does work. So, pretty much this error should equate to an AviSynth installation or plug-in directory problem or am I, no I am assuming this.


Also, sense we are here, I have seen I believe, in VDMod again.

env = CreateScriptEnvironment(2);
if(!env)
env = CreateScriptEnvironment(1);

and in avisynth.h (from 2.56 source)
enum { AVISYNTH_INTERFACE_VERSION = 3 };
If I remember correctly when "AVISYNTH_INTERFACE_VERSION" was used with my source last year (!env == true), so I went for "2". "Three" works in FilmCutter now(or at least today and yesterday) (AviSynth 2.57).

??????

Should an application do the if(env) thing and down one until true? Any other question I might ask is based on again, an assumption.

The conclusion, I can have my cake, and eat it.

I can display the message thrown to err.msg or catch the crash. Works for me.


Thanks,
Rob

[IanB]

IanB you wrote:
I guess you need to aggressively validate the err struct before you trust it.

I have no idea what you mean by that. I like aggressive, I would happily be that if I knew how. :)It means programaticly trust NOTHING! Check every pointer to not be NULL. Check every pointer points to valid memory. Check the value of every element for reasonableness and cross consistancy.

Again, I am not sure what you mean, unless ....
I was more thinking like thistry {
env = CreateScriptEnvironment(2);
}
catch(AvisynthError err) {
if(err.msg) {
try {
MessageBox(hwnd[0],err.msg,"AviSynth Script Environment Error",0);
}
catch(...) {
MessageBox(hwnd[0],"Opps! Error text corrupted!","AviSynth Script Environment Error",0);
}
}
else {
MessageBox(hwnd[0],"Opps! Error text NULL!","AviSynth Script Environment Error",0);
}
return;
}
catch(...) {
MessageBox(hwnd[0],
"AviSynth very seriously stuffed, Save your work!\n"
"avsFilmCutter could not create a script environment.\n"
"This means your AviSynth installation is screwed.\n\n"
"Go ahead, click Ok, I dare you.","AviSynth Crashed",0);
return;
}

Actually as I think about this I can see why err.msg is stuffed.

It is because of the use of SaveString not missed occurances!

The string space is part of the Env which is of course gets released on a prang. DOH!:stupid:

----------------------------------------------------------

If I remember correctly when "AVISYNTH_INTERFACE_VERSION" was used with my source last year (!env == true), so I went for "2". "Three" works in FilmCutter now(or at least today and yesterday) (AviSynth 2.57).
??????
Should an application do the if(env) thing and down one until true? Any other question I might ask is based on again, an assumption.This is a bit flakey. Interface versions 1 and 2 are incompatible, you need separate code to deal with each, 1 using Avisynth.h from the 2.0 stream and 2 using Avisynth.h from the 2.5 stream. Version 3 is a superset of version 2, and implements some extra calls like SubFramePlanar.

The code inside avisynth.cpp is this if (version <= AVISYNTH_INTERFACE_VERSION)
return new ScriptEnvironment;
else
return 0;And unfortunatly return an Env if version is 1, which is probably not quite right. It really should be returning a Version 1 ScriptEnvironment structure. So you need to poke for version 2 first to see if you need a 2.0 or 2.5 avisynth.h based code to understand the returned Env structures.

If you need to use the extra calls introduced in 2.5.6 then you must ask for Interface Version=3. If you do not then it is safe to ask for Interface Version=2.

[Ebobtron]

I see that my audition as forum lexer was a waste of time, as it seems to have one. The teacher, always sending me to the dictionary, “prang“, like “dross”, words not in common use on this side of the globe, or at least not my part of it. Although, prang does sound more like bending metal. Or, maybe I should report you for using language other that English in the forum. :) No, that will not be necessary, the rest of the post should get me in enough trouble.
Check the value of every element for reasonableness and cross consistancy.I shall master the divide by zero thing someday, it must be my mission in life, I have been doing it for some thrity years. I don't even try to do it and I do it. :eek: Check every pointer points to valid memory.I do not know how to do that. Back to pointer school for me (actually I skipped pointers, got an F).Check every pointer to not be NULLIs not that contradictory, I guess your going to tell me that a null pointer is not null (please see below).
Any attempt I make to access “const char * const msg” is met with a crash when
“MessageBox(hwnd[0],err.msg,"CreateScriptEnvironment",0);” is crashing.

When it is working this works too.
if(err.msg)
MessageBox(hwnd[0],”Unknow error”, "CreateScriptEnvironment ",0);
When not working this crashes also.

I suppose I did a poor job explaining, sorry.
The use of "if(err.msg)" crashes FilmCutter because, I am assuming, AviSynth crashed after throwing an exception during this statement "env = CreateScriptEnvironment(2);" contained in the try block.

When AviSynth does not crash when detecting an avsi file syntax error, "if(err.msg)" works just fine and the error message inside of "err.msg" is correct, except for the garbage.

http://members.aol.com/ebobtron/err_2.jpg
Unfortunately your "if(err.msg)" following your first catch behaves the same way.

Before I continue:Is not your second catch(...) meant to catch any other data type thrown other that AvisynthError, is that possible?

A catch(...) preceded by catch(AvisynthError err) is superfluous unless AviSynth throws exceptions with data types other than AvisynthError. Else why have the special AvisynthError class at all.

Of course I am assuming a few things. But I have proven the following in my application.
"catch(AvisynthError err)" always catches the exception (this being the source of my conclusion that AviSynth throws no other exception type)
"catch(...)" always catches the exception, except when preceded by "catch(AvisynthError err)"

More, I think is just minutia.

Is the following correct?

Using "err.msg" in an if statement, or as an argument for MessageBox() is not the cause of avsFilmCutter's crash. AviSynth crashing, or whatever, when encountering a avsi error and then "err.msg" becoming invalid is the cause of my crash, if not properly caught.

Does it have to be a crash or can a dll terminate without being released.

AviSynth with one error in one function inside one avsi file has pooped the sheets and run off crying. Sometimes though it tells mommy and then sits doing nothing.

Your job is much, much harder than mine.

My method

try{try{env pointer equals}catch(AvisynthError){opps "AviSynth detects Error"}}catch(...)(opps "AviSynth Stuffed")

works.

Here is more info concerning AviSynth's offputting lack of cooperation.

The list below shows the contents of my plugin folder.

AnSide.avsi
colors_rgb.avsi
descratch.dll
DGDecode.dll
DirectShowSource.dll
NicAudio.dll
TCPDeliver.dll
test.avsi
TransAll.dll
zoom.dll

Error anslide.avsi and AviSynth reports.
Error only test.avsi (the three line simple user function) from the prior post and she crashes.
Error anslide and test, reports the first only
add the file toom.dll (renamed copy of zoom.dll) prang.

Added other stuff, jpegs, txt files, only a renamed dll crashes with the syntax error in anslide.

--------------------------------
This is a bit flakey.I had not intentioned to support 2.0. I have checked interface with all versions of 2.5 I could find. And it seems my needs are met.

Nevertheless I sware when using 3 I crashed with 2.56. I was, aparently, wrong as I just downloaded and installed 2.56 and it works just fine.

All FilmCutter does is hook up and grab the function names. Later grab the function parameter strings, if they are available (which they are in 2.57, thank you, and Fizick). I also do a clip properties look up, independent of the directshow or AviFile interfaces.

I should do 3, if !, do 2, !1 ever. Or did I miss something.
-----------

So you speak Homer in Melbourne too. My wife's aunt in Coobowie said that was limited to the Sidney area. ? :) I don't think she gets out much.

And of course thanks.


:thanks:

Bye

[IanB]

... always sending me to the dictionary, “prang“, like “dross”, words not in common use on this side of the globe, or at least not my part of it. Although, prang does sound more like bending metal.Sorry for the coloquialism's, but you got the point and the Automobile crash reference :D

Check every pointer points to valid memory.I do not know how to do that. Back to pointer school for me (actually I skipped pointers, got an F).Okay, I threw you a curve ball here. This is actually very difficult to check. Volumes of MSDN articles have been written on how to do this. The short answer is wrap the access in a try {avagoyamug;} catch (...) {opps;}

Check every pointer to not be NULLIs not that contradictory, I guess your going to tell me that a null pointer is not null (please see below).Okay bad grammar here. The aim is to make sure you do not ever dereference the NULL pointer.

The use of "if(err.msg)" crashes FilmCutter because, I am assuming, AviSynth crashed after throwing an exception during this statement "env = CreateScriptEnvironment(2);" contained in the try block.The code generated by the compiler actually refers to "err" by its address, so technically it is a pointer which of course can, unfortunatly, be NULL or point to invalid memory.

So the test could be if (err && err.msg) which catches the NULL cases but not the invalid cases. Something about a maze of twisty passages that look similar.....

Before I continue:Is not your second catch(...) meant to catch any other data type thrown other that AvisynthError, is that possible?

A catch(...) preceded by catch(AvisynthError err) is superfluous unless AviSynth throws exceptions with data types other than AvisynthError. Else why have the special AvisynthError class at all.The catch (...) catches all and any other exceptions like GPF's and Access Violations (Well in VC6 at least, the newer compilers annoyingly do not, for that you now need the TRY EXCEPT syntax, which of course you cannot mix with try catch. Grrrr!)

Is the following correct?

Using "err.msg" in an if statement, or as an argument for MessageBox() is not the cause of avsFilmCutter's crash.It is the ultimate cause of your crash! Both err and err.msg have to be valid. And there is history here....AviSynth crashing, or whatever, when encountering a avsi error and then "err.msg" becoming invalid is the cause of my crash, if not properly caught.

Does it have to be a crash or can a dll terminate without being released.

AviSynth with one error in one function inside one avsi file has pooped the sheets and run off crying. Sometimes though it tells mommy and then sits doing nothing.

Your job is much, much harder than mine.Fundamentally there is an error in the way Avisynth is doing this.

Avisynth implements a String space as part of Env. Correctly used this allows text strings to be class level static but still reentrant. And because they are part of the Env object they all get released when Env is released. So we don't get memory leaks from text strings. But here we want to throw an AvisynthError object, fine, but the object has a pointer to a string which formally is within the string space of the Env which of course has just been release during the throw-catch stack crawlout.

So you see you are at the tail end of a 50 car prang on an interstate highway. Until Avisynth is fixed you need to fit active cruise control and ABS brakes and airbags and seat belts and etc to your code to avoid the fallout.

My method

try{try{env pointer equals}catch(AvisynthError){opps "AviSynth detects Error"}}catch(...)(opps "AviSynth Stuffed")

works.In my enthusiasm, I probably should not have removed the very outer try catch (...), this is the overall safety net! All the other code and loops and test are to try and isolate what is going wrong. Maybe you don't need this level of information.

i.e.
Did I get a valid Env?
Did I get an AvisynthError?
Is the err object valid?
Is the err.msg string valid?
Did I get another exception?
...
All FilmCutter does is hook up and grab the function names. Later grab the function parameter strings, if they are available (which they are in 2.57, thank you, and Fizick). I also do a clip properties look up, independent of the directshow or AviFile interfaces.

I should do 3, if !, do 2, !1 ever. Or did I miss something.In your case, you want something from Avisynth that is only correctly available in 2.5.7 so you should just ask for I.rev 3 and if (env == NULL) complain that the avisynth.dll is to old. You might also check the actual rev and limp along after a warning that 2.5.6 doesn't quite implement the needed features correctly.

In the general case you should ask for the lowest I.rev that supports your needs. Unfortuantly things didn't quite go right from I.rev 1 to I.rev 2 so you need some special stuffing around to support I.rev 1 and I.rev 2 together.

i.e. If you don't need SubFramePlaner, etc but need the general 2.5 feature set then ask for I.rev 2. All I.rev 2, 3, 4, etc avisynth.dll's will support those functions and return a valid Env.

If you need 2.5.6 features like SubFramePlanar, etc then you must ask for I.rev 3.

If you need 2.6 features like YV24, YV16, etc then you must ask for I.rev 4 (or whatever it ends up being)

[Ebobtron]

Ian,
Thank you very much; I think I can return to my little project with new confidence in my understanding of what I thought I was doing.

Without trying to beat a dead one.
FYI
The odd case of the AvisythError class member "msg" becoming invalid and crashing the catch block when it tries to read the string, seems isolated to AviSynth versions 2.56 and 2.57.

Versions back to 2.06 (I believe the last version without a version string) seem to keep the error msg pointer valid and report the syntax error correctly, without garbage.

2.56 may be valid or invalid, but no garbage and provides valid data for both the large and small avsi scripts. Renamed plugin.dll in plugin folder prompts invalid "msg" for large scirpt only, with 2.57 never having a valid "msg" with the small script and only reporting the large scirpt when renamed.dlls are not in the folder.

version 2.56 and prior report.

"script error: syntax error"
"(filename.avsi, line #, column #)"

version 2.57 reports

"pxxxpt error: syntax error" x = junk, mine are squares
"(filename.avsi, line #, column #)"

This is all I know.

Thanks
Rob

[IanB]

It's looks like what your saying is in all those memory leaks Tritical and I fixed from 255 to 256 we got a little over enthusiastic. I have added a note to my todo list to review these changes.

By all means review then code and report what you find.:cool:

Of course the annoying thing is it has taken over 2 years for someone to report this problem.:(

[tritical]

The whole problem here, as IanB stated before, is that env->ThrowError() uses SaveString which saves into memory allocated by scriptenvironment's stringdump object which, if the error is thrown while inside scriptenvironment's constructor, will be destroyed before you can access the msg.

Prescanplugins(), which parses avsi files, is called inside scriptenvironment's constructor. Prescanplugins utilizes the script parser which has many env->ThrowError() calls. In my testing with your anside.avsi file the initial error comes from

throw AvisynthError("Script error: syntax error");

inside:

PExpression ScriptParser::ParseAtom(void)

it then gets caught in

PExpression ScriptParser::Parse(void)

where it is immediately thrown. It is then caught again inside the same function and the following occurs in the catch():

env->ThrowError("%s\n(%s, line %d, column %d)", ae.msg, filename, tokenizer.GetLine(), tokenizer.GetColumn(code));

Where ae.msg is the initial "Script error: syntax error". Now the msg is stored via savestring and becomes corrupt when you try to access it later in your program. If the above line of code is replaced with:

throw;

then the original msg ("Script error: syntax error") is accessible just fine, as expected.

AFAICT, there isn't any way to fix this problem of accessing msg if createscriptenvironment fails except to change every place in the code that calls env->ThrowError() (and that could be run during scriptenvironment's constructor) to not use env->ThrowError(), change ThrowError() such that it doesn't use savestring(), or change savestring() such that it doesn't rely on memory allocated by a scriptenvironment member.

As for why it seems to work ok with older versions of Avisynth I don't know. All the way back to 2002 env->ThrowError() uses savestring(), and the scriptparser code hasn't changed since at least Nov 2003. Maybe you are just getting lucky (the pointer is pointing to freed memory but it hasn't changed)?

[IanB]

Tritcal,

I had the idea of wrapping the constructor code in a try/catch(AvisynthError) and copying the error text either to a static buffer or doing a strdup and wearing the small memory leak, then directly re-throwing the exception. Neither is perfect but it sort of covers the problem.

Thoughts?

[Ebobtron]

Well,I am at a loss, what question to ask next, I could ask far too many.

But first, my needs concerning FilmCutter's AviSynth interface have been met, so let me say thanks again.

I understand the invalid pointer returning garbage thing, my experience with FilmCutter is replete with garbage strings, garbage arrays. I have a small but growing bare spot on my head where I scratch while looking for my current “doh!”. My experience is still limited and my formal training is not.

If I understand:
... I can have a pointer to released memory, which I can read (correct or with garbage).
... I can have a pointer to memory in use that can cause a terminal access violation.

This comes to mind.
In the matter of the currently discussed syntax error in an avsi, file. The exception is fatal. Why throw a pointer, can we not throw the data? Am I trying to make this too easy?

I mean, like, the interface is toast, I cannot even read the version string, so I doubt that I might try any frame buffer work. Is a memory leak important at this point?

As I noted before, this type of error dumps direct show into the potty, where it stays until it is externally terminated.

Looks like I need a try catch thing around "pGraphBlder->RenderFile(wfile,NULL)".

More info:
To be fair IGraphBuilder does ok, reporting that it ran out of memory. IMediaDet's get_StreamMediaType() is the item locking FilmCutter up. It don't crash, it never comes back.

Don't you just love it.


Oh well, one thing at a time.

[tritical]

Tritcal,

I had the idea of wrapping the constructor code in a try/catch(AvisynthError) and copying the error text either to a static buffer or doing a strdup and wearing the small memory leak, then directly re-throwing the exception. Neither is perfect but it sort of covers the problem.

Thoughts?
Sounds like a good plan to me, especially since it's so simple. Having a small memory leak in that case seems acceptable if a static buffer is to be avoided.

[Ebobtron]

Hello,

Didn't think this needs a new post.
Recent work with AviSynth error detection suggested another issue I seem to have overlooked.

DirectShowSource can dump some big error strings I am guessing far in excess of two*MAX_PATH. This is not a complaint just a specific observation.

I don't want to just assume something, is there a max error string for AviSynth?
or
Will it always be a pointer to a constant?

:) :thanks:

Robert ... avsFilmCutter.com (http://avsfilmcutter.com)

[IanB]

I can't see any limit. It can be as long as any filter author wants to throw.

The error text is usually in the env->SaveString() space so will disappear when env is destroyed. (hence the problem earlier in this thread)

[Ebobtron]

I can't see any limit. It can be as long as any filter author wants to throw.

The error text is usually in the env->SaveString() space so will disappear when env is destroyed. (hence the problem earlier in this thread)

Ok, good.
I was afraid there might be some limit like
STRSAFE_MAX_CCH (2,147,483,647) or MAX INT or something.

:)