Crashbug (exception on unload) hopefully fixed

[sh0dan]

I have found, and fixed a bug on my own test-setup, involving AVI-source unloading, that would make it crash after first pass/unloading avisynth.

Please download and test from here (http://cultact-server.novi.dk/kpo/avisynth/avs_cvs.html).

If anybody else also get their crash-bug's fixed, I'll wrap it up and do 2.05 ASAP.

[hakko504]

Download:...Done
Test:...Reboot Done
Test:...File Opened in VD.
Test:...VD plays fine
Test:...Closing VD: CRASH D**n

[hakko504]

Sorry, problem remains.
:mad: :( :confused: :angry:

[sh0dan]

Yearh... D*mn....

Hmmmm... (Back to the code ;)

Anybody else? ;)

[sh0dan]

@hakko: Could you do a testclip for me - I cannot make a Matrox MJPEG myself, so if you could post a short clip here or mail it to me (kp[at]interact.dk), and include the script you use, that seems to be the only way to track it down.

[hakko504]

Script: no problem, my two template files are attached here.

Clip: Well... these files tend to be quite big: in fact the smallest size where I can guarantee that they are exactly the way I capture them is 50MB, i.e. when using VirtualDubs multisegment capture. Zipping won't help very much here. While I'm probably able to send a file of that size, are you able to recieve it? Or do you know a free (ftp) site where I can put the files while you get them?

[WarpEnterprises]

It seems that one error which I encountered (when closing the avi not the whole VD) has gone, I can't reproduce it anymore.

But as hakko writes the MJPG error stays (and it is really linked only to this because opening a huffyuy don't causes the error)
When using drwatson it says:

Überlauf im temporären Speicherbereich der Anwendung.
(=memory overflow in temporary memory of application)
Modul: KERNEL32.DLL
Beschreibung: Kernkomponente des Win32-Kernel
Version: 4.10.1998
Produkt: Betriebssystem Microsoft(R) Windows(R)
Hersteller: Microsoft Corporation

Anwendung: Virtualdub.exe
Beschreibung: VirtualDub video processing tool
Version: 1.4.10
Produkt: VirtualDub
Hersteller: Please suggest one. I haven't come up with a good one yet.

[sh0dan]

Originally posted by hakko504
While I'm probably able to send a file of that size, are you able to recieve it? Or do you know a free (ftp) site where I can put the files while you get them?

If you're able to send the clip, I'll be able to recieve it (100 Mbit connection). :)

[hakko504]

Actually I was more thinking along the lines of maximum filesize for a single e-mail.
On the other hand I hope I have found a place where I can put the files for you. I'll mail you the details as soon as I have a clip available for you. :)
Also let me say that I very much appreciate your help on getting rid of this strange error. I must say I still don't understand why it works with A2.02 and not with later versions. I have browsed the code and I can't find anything that should affect this, it's more like, either it shouldn't work in any version or it should work in all.

[sh0dan]

If you got problems with the place you're putting it, just PM or mail me, and I'll give you an FTP-account.

[sh0dan]

GOT IT!

Found out why segmentedavisource crashed. It tries to keep opening new AviSources until it fails - that creates an invalid object, that will eventually make AviSynth Crash, when it tries to deallocate memory that was never allocated.

Anyway, I uploaded a fix to CVS (see link above) - all worries should (hopefully) be gone now. I'll wrap it up, and if it works, I'll do a 2.05.

[hakko504]

Nice One!

I was about to post to say that I have a clip ready for uploading, and that my last post (Edit:removed) was wildly inaccurate as the clip that crashed does not have dropped frames...

Just give me a few minutes to test this new version and I'll be right back.

[sh0dan]

<crossing my fingers>

[hakko504]

Sorry, still crashing... I'm currently uploading the files to a server, and I'll send you an email as soon as I have verified that they are in the right place.

[sh0dan]

ok - thanks for the efford. I'll upload 2.05 anyway, since it contains a bunch of other fixes.

[WarpEnterprises]

Can again confirm and detail hakko:

1. Crashes if video is 3 frames long but only after actually viewing at least 1 frame (not when only opening AVS and not moving the slider)
2. Bug introduced in version after 29.7.2002 (the one from that day still works)
3. Crashes only on Matrox DMB1 codec files.
4. Crashes even when audio removed from AVI.
5. Does not crash when using ReduceBy2 or no resizing at all
6. Crashes when using BilinearResize or BicubicResize

[vinetu]

@WarpEnterprises
Have you test the matrox.avi files on another computer?
(with nonmatrox mjpg decoder)

[WarpEnterprises]

Yes, did it right now.
Changed the fourcc in the files with a hexedit to Picvideo and Lead, same crash.
(then at least sh0dan should be able to reproduce the crash).

I attach a 3-frame AVI with fourcc DMB1 which crashes on my PC.

Even when I re-encode the captured avi with the DMB1 the file crashes.
It does not crash when I reencode with Lead or Pic.

This would mean that the AVI is a little strange, just as much to crash 2.04 but not 2.02.

I remember that dividee(?) synced avisynth with virtualdub.
As I previous mentioned VD crashes when using the bulit-in MJPG-routines.
What about this connection?

[vinetu]

So, I've tested Avisynts 2.04(from CVS) with the ct3.zip avi and Mainconcept MJPEG
... same here - VDub stoping and waiting for something (or someone from THE GREAT ,
THE VERY ONLY, THE BEST (actualy the f***est) M****X corporation ) - I can't describe
this as crash because Vdub is just waiting ...at various frames (2,17,28...)
To compensate the 3 frames limit I use Segmented source (16avi files ->48frames):
--------------
LoadPlugin("D:\PROGS\AVSS\TomsMoComp.dll")
LoadPlugin("D:\PROGS\AVSS\SimpleResize.dll")
SegmentedDirectShowSource("D:\Temp\CT3.avi",fps=25)
TomsMoComp(1,15,1)
SimpleResize(320,288) # plugins are just for fan
----------------

or SegmentedAviSource.

..both scripts stops at various frames...

Next is the Avisynts 2.05 (from CVS) - I CANNOT reproduce the error !!!

sh0dan !

I'm sure it's DONE !!!

comp:
WinXP(clear instalation),P4,Matrox FREE machine ...

Best Regards!

vinetu

[sh0dan]

I think I have it reproduced.

[WarpEnterprises]

I did a test on a w2k system with no matrox and the picvideo codec to decode the dmb1: no crash.

[sh0dan]

@hakko: The clip you supplied does not contain any valid frames in segment 3 (test1.02.avi), (both frames are dropped) so that is why it fails.

Avisynth must fail in this case, since it cannot determine any valid frames for frame 0. You have to see the segments as seperate AVI's that are spliced together.

I've added error-reporting, so you can identify the error in the future (AVISource: could not find valid keyframe for frame 0).

I've also removed all checks for errors, so Avisynth will now return the last valid frame, if any errors occur.

I'll upload a test version to CVS-bin-place.

[hakko504]

@sh0dan

Interesting, but I have two main questions that casts a slight doubt about your answer: Why does it work when you use Reduce instead of bicubicResize?
Why does it work in 2.02?
About test.02 beeing a null file: that's the way VD handles multisegment capture, it always creates null files ahead of time to make it easier when it is time to change file. You can probably find more info on that subject in VD's help file and on Avery Lee's VD site. (http://www.virtualdub.org)
Final note: Have you tried Warp's 3-frame-long file yet?

[hakko504]

And of course I'll test the new version as soon as I can (with/without the last file), but don't expect report on that until late saturday or early sunday.

[sh0dan]

1. Win98 is not the best at detecting memory violations, when they actually happend. Win2k fails immidiately, when
the cache is trying to access the picture.

2. Perhaps 2.02 already fails at fileopen, when frame 0 is invalid, that would make SegmentedAviSource stop if frame 0 is invalid. I'll try to see if I can implement that.

3-frame file works fine (with 2.05+ (23-08-2002)). It uses Morgan MJPEG on this machine to decode it. It could very well be fixed in the latest binary, and not be codec-specific.

EDIT: Clarifications

[hakko504]

@sh0dan

You are so 100% not going to like this: IT STILL CRASHES. I just changed to 2.05+ and I removed the offending empty file. No luck, VD still crashes on exit when I use bicubicresize instead of VerticalReduceBy2(). As soon as I have rebooted the machine I will look at Warps short file myself.

[hakko504]

@WarpEnterprises

What's your experience with the new versions?

[hakko504]

Originally posted by WarpEnterprises
Can again confirm and detail hakko:

2. Bug introduced in version after 29.7.2002 (the one from that day still works)


@sh0dan

The resizing functions were changed on July30th. Could you mail me a special version of AVIsynth where you have reverted the file resample.cpp to version 1.2 instead of the current 1.4?

[sh0dan]

There is a special version on the bottom of the CVS-bin-page for you, my friend. :)

I would be a big surprise for me, if the july 30th changes causes crashes for you, but it's worth a shot.

[WarpEnterprises]

No crash with the special version.

[sh0dan]

... and the 2.05 at the top of the page does?

[hakko504]

Originally posted by WarpEnterprises
No crash with the special version. Same result here:
No crash with the special version.

:) :D :cool: :) :D :cool: :) :D :cool: :) :D :cool:

I see 3 possibilities here: MatroxMJPEG doesn't like __declspec(align(8)) at all. It might accept __declspec(align(16)) There is something wrong with the __declspec or align functions that might create more problems later.

[hakko504]

Originally posted by sh0dan
... and the 2.05 at the top of the page does? Dies as soon as you exit VD.


And a large thank you sh0dan (and the rest of the AVIsynth team) for the efforts you put into this great application. It seems that you have found a number of minor bugs that could have become problems later on if they hadn't been corrected now.

[sh0dan]

Strangeness - I have no explanation whatsoever - The variables should not affect anything outside the FilteredResizeH and V. Could be a compiler error, since they are linked statically to a direct memory address according to the disassembly.

Do you have any similar problems with layer or mergechroma/mergeluma?

Well - great work in finding the bug! It was a tough one. I'll remove all these aligns, until we have found the reason for the problems with them, and commit it - hopefully this weekend.

(this bug is really strange - could win98 have a bug in memory allocation when loading the dll?) :confused::confused::confused:

Anyway, we should make a routine that is capable of returning 8 or 16 byte aligned __int64 pointers - I think I even discussed this with dividee recently.

[hakko504]

Originally posted by sh0dan
Do you have any similar problems with layer or mergechroma/mergeluma?
Sorry, I haven't tried those functions at all. I try to keep my scripts as clean as possible. Most of the time I just use a de-interlacer and a resize, sometimes a little denoising. I mostly find other stuff too timeconsuming. (I really want a Dual Athlon system...)
Well - great work in finding the bug! It was a tough one. I just tried some simple reasoning: It works with reduce, but not with resize. Then when Warp pinpointed the start of the crashes (July 29th working, later didn't) I looked at the files that were affected by those functions and found that resample had been changed on July 30th, whereas resize (that contains reduce) hadn't been changed at that time, I thought, why not try the old version and see if that work. And it did. Yeah!
(this bug is really strange - could win98 have a bug in memory allocation when loading the dll?) :confused::confused::confused:
There are a lot of bugs in all windows versions...

[sh0dan]

Ok, removed them from the latest code (with the saturation bug-fix), and made some code alignments - there's a "test 2" I'd like you to take - it should work OK now, but I'd like to be sure, if you understand! :)

[hakko504]

Test2 also seems to work just fine. :)

[dividee]

great work guys!
I just did a quick search on the web but didn't found much info about this __declspec(align(8)) issue... except one comment (http://www.tek-tips.com/gviewthread.cfm/lev2/4/lev3/29/pid/116/qid/340878) in a forum that says that __declspec does not support 'align' in VC6... and in the xvid changelog (http://umaniac.leffe.dnsalias.com/alpha/changes_full.txt) 25/3/2002: "removed __declspec(align(..)) for _MSC_VER <= 1200"

[sh0dan]

Well, now they're gone in "resample" (Bicubic/Bilinear resize).

Anyway, would either of you test "MergeLuma", "MergeChroma", "ConvertToYUY2" and "Layer" - it's also present in these filters, and I would be very unhappy to have to remove them from ConverToYUY2 at least, since it would give a large speed penalty - the others doesn't matter as much.

Edit: Added new audiofunction today (EnsureVBRMP3Sync), to get rid of my bad conscience for making this bug. :)
Edit2: .. and extensions to letterbox - now I feel good again! :devil:

[WarpEnterprises]

Merge, Layer, ConvertToYUY2: can't produce an error.

[sh0dan]

Cool! Thanks again!

[hakko504]

@sh0dan

BTW, with the new version, (test2 from last night) I get an error (not crash) when AVIsynth tries to read from the empty file at the end of the capture. It shouldn't affect linear encoding, but I wasn't able to decode any other frame after that. Removing the offending file of course solved that problem.

[sh0dan]

You should also get that, when frame 0 is invalid. I'll investigate how to most gracefully handle this situation - the old way seemed to work, but it returned a null-pointer as a frame, which could lead to crashes elsewhere.
[edit]
The best thing would be for the filter to fail on load, and then SegmentedAviSource should catch that, and not add the file at all.

[WarpEnterprises]

After a while I used a captured VD seqmented AVI again, and there still is a strange buggy behaviour probably caused by the last (and always empty) segment:
- VD crashes sometimes on exit
- TMPG can open the file only every 2nd time, the other time it reports "unrecogniced exception" (as an Avisynth error msg in the output video).

@Hakko, sh0dan: did I miss something or is this bug still present and can only be solved by deleting the empty segment?

[You stated once:
Notes:
2.05 Known bugs:
- Segmented captures may lead to crashes, if the last segment isn't a valid AVI file (that is, doesn't contain any frames). Try version 2.02 instead.
]

[hakko504]

The behaviour of (Segmented)AVISource has been changed slightly in v2.05+ and the last empty segment created by VD when capturing must be deleted manually to avoid AVIsynth crashes. Every file opened must be valid and the empty one isn't. Since I started deleting the last file I have not had any problems with the newer versions of AVIsynth.

[sh0dan]

Having an empty file as the last file shouldn't be a problem in 2.06 (at least with the test you sent me). If any segment fails to decompress frame 0, that segment is just skipped.

[WarpEnterprises]

Sorry, but there seems an error left.
I put a small sample here (http://members.aon.at/archi/wepictures/cap.zip)
PicVideo codec, captured with VD.

If opened via SegmentedAviSource in TMPG, every 4-5 times (only!) it displays "SegmentedAviSource: decompressor returned error: AviSource: Could not decompress frame 0! (filename)
The other times it shows a black video or works correctly.

With the lines you wrote in mind it seems that SOMETIMES the empty segment is NOT skipped, which then causes errors or even crashes.

I will test this on a W2K machine tomorrow.

[sh0dan]

Thanks! I'll look at it!

[WarpEnterprises]

I tested on a W2K.
There it always works, but the number of handles (in the taskman don't know what that is exactly) rises by 10 each time you reopen the file (with TMPG or VD)
But maybe this is normal, although it doesn't occur when reopening a plain AVI.

[WarpEnterprises]

Bumping a quite old problem, that I still encountered and other people wrote about it too:

With different sources there is the following error:

In VD every second time the AVS is opened there is "can't decompress frame 0" - even if it is from mpeg2source.

The error seems to depends on
- the source video
- the script commands e.g. resizing to different sizes

Any ideas how to pin it down?

(I'm going to test further different versions and videos).

[sh0dan]

Yeah - it's not uncommon. For experiments sake I'll remove the "frame 0" test - I'm not sure if this will result in unexplained crashes - but disabling it until the next beta would be helpful!

I'll make some noise, when a test version is ready!

Edit: Got it to compile - Removing it seems to fix a "could not decompress frame 0" i have on RAW RGB24 AVI's.

[WarpEnterprises]

My version is the latest 2.0x.

But when I understand you correctly it's independent of 2.0 / 2.5 ?

Can you please explain a little more detailed where the error origins?

And why can the error only be seen every second call (ie refresh in vdub-mod) ?

[sh0dan]

Honestly I have no idea. The stupic thing about this is that I can only see the results of the changes, not the causes - Now I know why Avery swears so much! ;) (quote from Ben RG - now it also gives meaning to me ;-)

[sh0dan]

ok - I think I actually have a "fix-candidate" - without disabling the current code.

Some debugging revealed a potentially uninitialized variable being checked.

Again - I'll make some noise when a binary is ready.

(I'll merge the changes down into 2.0 branch also)

[sh0dan]

There is a Release Candidate for 2.08 on the CVS page.

Give me a go, and it'll be released ;)

[sh0dan]

@WE / hakko: How's your test going?

[WarpEnterprises]

@sh0dan:
Sorry, nothing tested yet (weekend is coming :) )

1) Do I see it right that the bugfix is only in your compile and the MAIN CVS branch which now is the 2.5branch ?

2) It seems that now is the time to officially stop adding functionality to 2.0, only bugfixing (i.e. no Turn, ...) . Do you agree?

[sh0dan]

2.0 has been updated:

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/source.cpp?only_with_tag=AviSynth-2_0

Don't know why it isn't shown correctly if sorted by date.

New features are not put into 2.0 - maintenance only! It has been so since 2.5 development started.

[hakko504]

Equally bad news here: haven't captured anything since before christmas - and about a month ago my Win98 gave me a BSOD and the message 'Please reinstall windows' so I defenestrated Win98 and installed Win2k instead. And just about a week later I got a DVD-R burner and I've been busy learning that one so far. And even worse: I'm proably going to work out of town for the rest of the spring, meaning my time for capturing/video encoding will be extremely limited in any case until after summer. You'll have to work this one out together with WarpE.

[WarpEnterprises]

Silly error this is. I wanted to test it with the new version, but couldn't reproduce it with the old one before (same source, script but did some of the usual install,... on the PC).
Anyhow the new version worked OK.
(only the installer seems to be buggy in associating AVS to notepad - it got ass. to WinZip, running it twice and it was ass. to Notepad again).

[matrix]

Need more reports?
Here it goes.
I had problems all the time with all releases after 2.05 including that. The usual error was 'could not decompress frame 0'
I downloaded a few days ago the release candidate, and it works beautiful now.
Just like 2.03, which is what I used up untill now, plus the extra features.
I'm happy with it.:D