Xvid and VirtualDub

[Zach]

I posted a detailed post over on the VirtualDub forums (http://forums.virtualdub.org/index.php?) before I saw it was actually against the rules to ask about Xvid crashes (:o), hence I asked for that thread to be deleted and here I am appealing to you guys. :p


I'll keep this simple. If you have the following installed:


VirtualDub v.1.7.6 (or 1.7.5, or 1.6.19)
Xvid (Koepi's 1.1.3 or Celtic Druid's 1.1.3 [or even his cvs.head.MTK version])


...can you create a test video (in VirtualDub, "Tools-->Create test video...-->RGB") and save it using Xvid as the encoder? (profile doesn't matter as I have tried most of them, but for conformance, use the "Advanced Simple @ L5" or "Home Theater NTSC.")

Regardless of what I try to encode, VirtualDub always crashes because of a memory access violation inside xvidcore.dll. On the other hand, Xvid encodes fine and without error in TMPGEnc 4.0 XPress (same source, same profile settings, etc.).

The reason this is so puzzling is that it worked fine a couple of weeks ago when I was doing some limited benchmarking (as I discussed in my Disillusioned and Confused -- DivX vs. Xvid Encoding Speed (http://forum.doom9.org/showthread.php?t=130678) thread), but then I got busy with other stuff (life, work, etc.), and now two weeks later when I try resume my benchmarking, I sadly find that VirtualDub no longer likes Xvid! :devil:

Given this, I know, most likely, the problem has to be on my end and not with VirtualDub, but I just don't know what else to do to track down the problem.

[Guest]

I just don't know what else to do to track down the problem. The source code for both is available. Fire up your debugger and trace the cause of the error.

[Zach]

That's true, I suppose, but it's always usually a pain setting up someone else's software environment, much less two separate projects (VirtualDub & Xvid). :)

Admittedly, VirtualDub would probably be easy and take all of ten minutes to get it up and compiling since he migrated over to Visual Studio a while back if I recall correctly, but I'm not sure about Xvid.

[Guest]

It's all a question of how badly you want to solve this problem.

[Zach]

So no one here can take the 30 seconds to save VirtualDub's test video to an Xvid file like I described in my OP so I can confirm the problem is on my machine alone and not with VirtualDub?

[Guest]

I tried it and it worked just fine. I have XP service pack 2, VirtualDub 1.7.6, Xvid 1.1.0 beta 1 (Koepi's build). Sorry, but I don't want to install anything else to run your test.

[Zach]

Okay, thanks! *goes off to hunt down the source code*


Rapid Edit: Where can find Xvid 1.1.0 beta 1 (Koepi's build)? I haven't tried that (older) build yet.

[Koti]

No crash , XP sp2, VirtualDub 1.7.6, Xvid 1.2.-127 (Oct-25-07) and Xvid 1.2.-127 from July-2006 as well

[fccHandler]

Admittedly, VirtualDub would probably be easy and take all of ten minutes to get it up and compiling since he migrated over to Visual Studio a while back if I recall correctly, but I'm not sure about Xvid.
I found Xvid a little tricky to build the first time, but once you've done it it's easy. I don't know if you can build it with VS2005, but it builds fine in Visual Studio 6. Nowadays I only use my own build of Xvid, so I can't help you with those other builds. (But FWIW, I did your test and it didn't crash.)

The source code I used is here:
http://www.xvid.org/Downloads.43.0.html

They also link to some binaries on Xvid.org, but they don't say much about them:
http://www.xvid.org/Downloads.15.0.html

You could also try deleting the registry keys:
HKEY_CURRENT_USER\Software\GNU\XviD
and
HKEY_CURRENT_USER\Software\Freeware\VirtualDub

That will cause Xvid and VirtualDub to revert to their original default settings, but you'll lose all of your saved settings (in particular, VirtualDub's capture settings). You might want to back up the keys first.

[squid_80]

Turn off "Display Decompressed Output" under the view menu.

[Guest]

I just repeated the test with Display Decompressed Output enabled and it crashed!

[squid_80]

Tada!

[Guest]

You be da man, squid_80!

[Zach]

Turn off "Display Decompressed Output" under the view menu.

:thanks:

No more crashes! I must have ticked that option on and not realized it.


So out of curiosity, though, with the option off, what output is being displayed?

[squid_80]

With the option off you see the video that's being fed to the encoder.

[Zach]

Well, that sucks. So you mean, basically, the output pane is just a copy of the input pane? (Edit: With the presumption that I'm only using VirtualDub to transcode, not to do any filtering.)

I guess I'll just stick with DivX then. I mean, that's the primary reason I use VirtualDub so I can easily see in real-time the "before and after" of an encode so I can more readily adjust the encoder (profile) settings, tweak the bitrate, etc.. :(

[Guest]

You've got the source. Be a nice chap and fix it for all of us. :)

[fccHandler]

Hmm, my test didn't crash when I turned on "Display Decompressed Output." (In fact, that was the first thing I thought to test.) So before giving up on Xvid, I suggest building the codec yourself, or perhaps trying that build they link to at Xvid.org.

[squid_80]

It doesn't happen when I use virtualdubmod, but started happening somewhere along the line with the 1.6 virtualdub series.

[Zach]

So before giving up on Xvid, I suggest building the codec yourself,...
I might just do that, especially since you don't get the error with your private build; but on the other hand, since I don't know beans about video encoders, tracking down the bug may be more trouble than it's worth. <shrug>

..., or perhaps trying that build they link to at Xvid.org.

Yeah, that's Koepi's 1.1.3 build, the one I was originally using when all this started. :p

I got it from codecs.com (http://www.free-codecs.com/Koepi_XviD_download.htm), but the download xvid.org links to (Digital Digest (http://www.digital-digest.com/software/download.php?sid=1052&ssid=0&did=1)) is bit-for-bit identical.

[fccHandler]

Disclaimer: I don't make a habit of distributing my Xvid builds, so this is just a one time thing. I've uploaded a copy of my build of 1.1.3 here:

http://fcchandler.home.comcast.net/Xvid.zip

You'll probably want to uninstall any other version of Xvid first. Then after unzipping the files, right-click "xvid.inf" and choose "install." That installs the VFW codec. To install the DirectShow decoder, run "regsvr32 xvid.ax."

I expect that some profiles you are familiar with may be absent from this build. I'll just say that this is a 100% pure Visual Studio 6 build of the stable 1.1.3 source from Xvid.org. I didn't change anything in their source.

[Guest]

That one crashes for me also if Display Decompressed Video is enabled.

[Zach]

Okay, I'll test it out now and see what happens. Will report back in a few minutes.

Edit: Well, neuron2 beat me to it, so never mind. ;)

[Zach]

@fccHanndler,

What version of NASM are you using? I grabbed the 0.99.05 version from sourceforge.net (http://sourceforge.net/project/showfiles.php?group_id=6208&package_id=47034&release_id=547669), but it balks when it tries to compile any assembly file that uses a 'dword' modifier as part of the address--which there are many :p. Also, oddly, it won't accept an include path (i.e., -I<path>)--when used, the rest of the command-line seems to get chopped off because it complains about not having an input file. (And no, it's not a 'space in path' type of issue.)

[fccHandler]

What version of NASM are you using?
Good question! I have no idea; it doesn't give a specific version number anywhere. FWIW, almost all of my NASM files are dated April 2002.

It doesn't really matter though; if my posted build of Xvid crashes on other systems, then there's probably no point trying to duplicate my build. :(

[Zach]

Just type "nasm -v" to find out what version you are using. ;) But at any rate, I've gotten past (i.e., fixed) all the assembly syntax errors, and now I'm just trying to work through the C errors.

It's just a matter of translating the project defines (NDEBUG;ARCH_IS_IA32;WIN32;_WINDOWS;ARCH_IS_32BIT) into their VS2005 equivalents (with some additions, of course). (Currently, VS is defining _W64 which is causing all the errors.) It should be simple to work out.



And yeah, I know I don't have to do this, but once I start on something... :)



Side-note: Incidentally, are you using a 'private' build of VirtualDub as well? If so, then 1.) this might not be an xvid issue but instead a VDUB issue, and 2.) it would explain why you are the only one (in this thread) not seeing the crash.

[fccHandler]

Aha! Here you go:

nasm -v
NASM version 0.98.36 compiled on March 13, 2003

And no, I did your test with the unmodified VirtualDub 1.7.6.

[Zach]

fccHandler, you should try encoding something external, preferably an Avisynth source.

I got both DLLs (xvidcore.dll and xvidvfw.dll) built and installed, but VirtualDub still crashes for the most part. With my versions, I was able to encode VirtualDub's test video 50% of the time, but anything else I tried to encode crashed 100% of the time. Odd.

I don't know. I guess in the next couple of days I'll go grab the VirtualDub source so I can run a debug build and see if it's anything obvious. (Public Disclaimer: If VirtualDub ends up taking more than half an hour to setup and compile error free, I'll probably give up this endeavor. :p)

[foxyshadis]

It won't with VS 2005, since the build environment has shifted to that, but older studios might give you trouble. The instructions are pretty clear. You'll have to download and install some things like directx sdk (I had that for MPC anyway), but that's a fire-and-forget type of install.

[squid_80]

I'm about 80% sure it's not a virtualdub problem since I'm positive it used to work with virtualdubmod and now it doesn't, and vdubmod hasn't changed at all. My gut tells me it's something to do with the codec attempting to reference previous frames that it got from the host app (vdub) which have been freed in the meantime. Disabling b-frames for example seems to make the problem go away.

Edit: So far I have found one unrelated issue with vdub - it doesn't check if display decompressed output is enabled until after writing the first frame, hence you get the "preview will resume starting at the next keyframe" message until the second keyframe is output. This can be fixed by moving some code around, I'm not sure if it's an oversight or if Avery has a reason for doing it that way.

[fccHandler]

fccHandler, you should try encoding something external, preferably an Avisynth source.
I have. In fact, I do that all the time. :D

I don't normally use b-frames though, plus I've tweaked my Xvid settings in other ways. So today I tried pressing "load defaults" before doing your test, and... it crashed! Wow, that's a first for me.

Just thought you'd like confirmation.

[squid_80]

I got it... virtualdub doesn't wipe the compressed data buffer clean between frames, it just gets overwritten again and again. xvid reads more of the input buffer than it should when trying to decode. Because of this xvid sometimes misinterprets junk left behind in the data buffer from the previous frame as data for the current b-frame that it's trying to decode and that causes the crashes. Xvid is really at fault but it's trivial to put a patch in virtualdub to fix it.

[Guest]

Why not put the patch where it belongs?

[squid_80]

Here's the xvid patch, I'm not completely sure it's correct though.

mbcoding.c:
int
check_resync_marker(Bitstream * bs, int addbits)
{
uint32_t nbits;
uint32_t code;
uint32_t nbitsresyncmarker = NUMBITS_VP_RESYNC_MARKER + addbits;

if (((BitstreamNumBitsToByteAlign(bs)+BitstreamPos(bs))+7/8) <= BitstreamLength(bs)) {

nbits = BitstreamNumBitsToByteAlign(bs);
code = BitstreamShowBits(bs, nbits);

if (code == (((uint32_t)1 << (nbits - 1)) - 1))
{
return BitstreamShowBitsFromByteAlign(bs, nbitsresyncmarker) == RESYNC_MARKER;
}
}
return 0;
}

[Guest]

How about giving us a binary? :)

[fccHandler]

I could be wrong, but I think "7/8" gets evaluated first and resolves to (int)zero. (Division has higher precedence than addition.) So I think you need one more set of parentheses, i.e.:

if (((x + y + 7) / 8) <= z)

Or is it:

if ((x + (y + 7) / 8) <= z) ?

Also, should it be "<=" or "<" ?

[squid_80]

It's not correct. This should be better but I'm still not sure because I don't know what all this RESYNC_MARKER stuff is all about.
int
check_resync_marker(Bitstream * bs, int addbits)
{
uint32_t nbits;
uint32_t code;
uint32_t nbitsresyncmarker = NUMBITS_VP_RESYNC_MARKER + addbits;

nbits = BitstreamNumBitsToByteAlign(bs);
code = BitstreamShowBits(bs, nbits);

if (code == (((uint32_t)1 << (nbits - 1)) - 1))
{
if (BitstreamPos(bs) + nbits < bs->length*8)
return BitstreamShowBitsFromByteAlign(bs, nbitsresyncmarker) == RESYNC_MARKER;
}

return 0;
}