VDub direct stream copy increases filesize. Explanation?

[pomo]

Hey Guys, some very strange situation.
I've got a Movie (DivX3-video(seems VBR); CBR-128kit/s MP3-audio, 667MB-size) which wouldn't play in WMP6.4 but played in DivX Player 2.0 Alpha (though became asyncronous towards the end). What I do at such a moment is have VirtualDub 'direct stream copy' it (usually solves lots of problems).

Now comes the very (!) strange part:
The resulting file of that had a size of 767MB (which is 100MBmore than the initial file :confused: ) and this version played syncronous in DivX Player 2.0 Alpha! (though WMP6.4 still didn't play it).
What I am looking for by now is an (logical and/or mathematical) explanation for the Movie increasing about 100MB in filesize though having done no recompressing (just direct stream copying it) as this is something I have NEVER seen before (though having some experience).

(More Details: WMP6.4 played the file when stripping away the audio and the audio alone played well too)
PS: I'm not looking for a solution (I'll probably just put that movie on an 90min disc, and I don't really care about WMP6.4 not playing it which is probably about codecs or bad avi-muxing).

So "How can an AVI increase in size by direct stream copying it?" (this seems very interesting to find out)

[alexnoe]

It can be slightly increased, such as by making smaller audio chunks for MP3. This could cause about 20 MB for a 2h film with 3 languages in MP3...

100 MByze is far beyond what this issue can cause.

If it is asynchronous to the end even if you don't zip around, then there's no reason to remux your file. You can simply adjust the frame rate and fit the length of the video to match it the length of the audio.

[BeNBeN]

This may be some "Headers" re-writing while dubbing probe.
Well, "headers" or key-frames (dunno the appropriate term).
Try to slide-search in the original file and in resulting one and see if there's a difference.

This is the only answer i'm afraid to have.

U can try to re-dub it too and get sure to select "Direct Stream Copy" for both V and A. Maybe u selected "Fast Recompress"the first time.

A+
BeN

[pomo]

@all
Well it's very shure I only did "direct stream copy" (video and audio)and nothing else. (no editing, no zipping, etc.)
But to be sure and get more information, I direct-stream-copied my original a second time and there was someting strange happening:
Vdub shows some 'project filesize'(while copying), well it displayed more or less 667MB until one point when fps went down from about 1000fps to 0.0 fps. After some time of nothing happening it continued streaming but project fiesize jumped to about 930MB and from then on decreases slowly to it's final size of 767MB. (strange :confused: )
Further details:
- the movie is called Contact and is 2h14min long
- Keyframes seem to be same in original and new file. (I checked some at the beginning of the 2 files and some at the end)
- VDub sais the number of frames in both versions is the same (so is the number of Keyframes)
- 'AVI File Info' sais that the original File has about 14000 frames more than the new one :confused:

@alexnoe
I did no zipping around and the asyncronous-thing is that the original movie is asyncronous (by original I mean original AVI-file ;)) and the 'direct stream copied'-one is syncronous.

@all
thank you for replying ;-)
PS: there is another thread running about just the contrary (VDub decreasing filesize when direct stream copying, but this seems having nothing to do with this problem)

[Swede]

Vdub shows some 'project filesize'(while copying), well it displayed more or less 667MB until one point when fps went down from about 1000fps to 0.0 fps. After some time of nothing happening it continued streaming but project fiesize jumped to about 930MB and from then on decreases slowly to it's final size of 767MB. (strange ) The only thing that comes to my mind is that your original file is corrupted since, like alexnoe said, 100MB is *way* too much for just index/chunk issues.

[alexnoe]

What does AVI-Mux GUI say about the frame numbers?

EDIT
if you trifle with the index of the file in a hex editor, then you can make a sequence being played twice, even if it is only once in the file.
VirtualDub wouldn't support such crap and would store this peace twice into the file (NanDub and AVI-Mux GUI don't support such crap either).
But I'd need a copy of the whole file to find out if this is indeed the case, or i'd have to implement a check for double-used frames into AVI-Mux GUI. I think I'm going to add this, but it will take some time to check for this with reasonable performance.

[pomo]

I've got no time by now to check about what aviMux GUI sais (I'll do this some later, but it'll take some days till an answer comes because I don't go online regularly).

@alexnoe
I'm sorry it won't be easy to give you that file as I don't have any flatrate at all. (perhaps I have afriend share it =complicated)
I also don't know where this file comes from as I got it from a friend (think he downloaded it in kazaa, not sure.)

I've got to go offline by now...

[alexnoe]

OK, I'll make up something to check the index of an AVI file for double occurences of entries.

[pomo]

@all
Back again... I couldn't wait to give an answer...
I took some time as I said and, well as the results of Vdub and AVIMux_GUI would take to much space in this thread I decided to upload them on my webspace...

http://polmolitor.php.lu/uploads
(find results of AVIMux_GUI's INFO and one screenshot of AVIMux_GUI crashing and two of VDub jumping that 'project file size')

@alexnoe
by re-reading this thread I now understand your first approach to my problem and I'm sorry I didn't understand it at that time but hought you were wrong. (I would also try this, just recommend me a prog to change the fps because AVIMux_GUI crashes (I bet you don't like to hear that :D ) and Vdub will change the stream anyhow.)

[alexnoe]

As to your pictures:
<quote> BTW: ... You are Alexander Noé, creator of AVIMux_GUI (or not?) </quote>

Indeed...

I would also try this, just recommend me a prog to change the fps because AVIMux_GUI crashes

As to the crash when trying to change the framerate: I did some changed to fix some memory leaks. Unfortunately, I forgot to make another change at another place of the code :( it is fixed now (you used the pre-release, right? Just redownload it, but make sure that it is not in your cache...)
Note that the idea of changing framerate was only for repairing an asnychronous-running movie...it won't help in any way with the strange stream sizes.

You're right: The value which you indicate being "strange" is complete nonsense.
The avg data rate is just calculated "size of stream / time", so a nonsense value for "size of stream" will lead to a nonsense value for avg data rate.

Since the video has been shortened by VDub a bit, the smaller audio stream is OK.

I'm pretty sure that your index block of the file is messed up.

I'll add more info to the info-dialog...the first one will be the AVI-Mux GUI version number, so that I don't have to ask silly questions as "which version..."
btw the german homepage of AVI-Mux GUI usually contains also the latest changes in the pre-release.

Done. It looks like this (http://www-user.tu-chemnitz.de/~noe/Temp/lotr.txt) now.
Please check if AVI-Mux GUI now finds strange chunk sizes (it is definitely strange if it finds chunks over 1 million bytes).

I think I know how to fix it. I'll add an option such as "ignore all chunks that are bigger than xxx kBytes". Will take some time, but it will exist somewhen...

EDIT
The newest pre-release can now try to circumvent bad index chunk sizes by trying to read the correct size in the stream. If this failes, a dropped frame is introduced. Note that deleting a deltaframe will mess up your video stream until the next key frame comes. Call "hinzufügen - erweiterte optionen", set the max size to 1000 and check "Reparatur versuchen". Then open your file (don't open it before!), which is now possible via drag&drop.

[pomo]

hi, I'm back again
Sorry it took so long, I don't go online that often, but that'll change in some weeks when starting studies at university of Kaiserslautern and finally getting DSL.
Then it will also be possible to provide you (alexnoe) that strange movie-file).

I'm downloading your AVI-Mux_GUI-1.10 by now (that pre-release section didn't contain any file so I guess 1.10 is the latest update).
(results of a test with it will follow...)

[off-topic]
as you are from germany, please tell me what DSL provider is best to choose
(most important is: flat-rate, no volume-limitation, cheap
still important is: low ping, no auto-disconnection
nearly unimportant: downstream and upstream speed)
don't know if you bcan help me there but asking doesn't hurt (or does it:confused: )
[/off-topic]

[alexnoe]

Yes, I've uploaded 1.10 final yesterday.

OFFTOPIC As to your traffic problem: I'm connected to university network. When downloading something from microsoft, 3 MB/s is normal :) /OFFTOPIC

[pomo]

hi, I'm back (as I finally got adsl)
I tried the aviMux v.1.10 out but twice it resulted in a file of about 271MB wich was unable to play (not even virtual dub could handle the resulting file).

I attached the inforation avi mux 1.10 gave.

(if you want to get video-file, I can provide it to you, just tell me then)
Otherwise I think we may give up finding out what this is all about.

[alexnoe]

I don't see any attachement...

BTW the latest is the english pre-release :)

[pomo]

ok, as far as I understood the point about attaching files, they have to be validated by a moderator first,
well, here's a link anyway: .../avimux1.10_info.txt (http://www.geocities.com/pol_molitor/avimux1.10_info.txt)

I downloaded your newest version an here is a link to its results:
.../avimux1.10-eng_info.txt (http://www.geocities.com/pol_molitor/avimux1.10-eng_info.txt) ;)

Guess what happended: same results for the file, it stopped at 271MB and aviMux closed automatically. :(

[alexnoe]

OK, the index of your file is definitely complete crap. What did you set in the advanced options as maximum chunk size?

[pomo]

nothing, should I have set something? (You're talking about aviMux I guess).
all settings I changed (others left on standart) was once dml-ouput an the other time I set the max user file to 900MB (to make shure the file fits)

[alexnoe]

Set the max allowed chunk size to 1000 kByte and have it tried to repair it

[pomo]

I did and AviMux didn't crash but put out a file 5 MB smaller than the original and even played in media-player6.4 (though the sound is asyncrone at the end towards the movie, at the beginning, it is all right).
here is a screenshot (http://www.geocities.com/pol_molitor/screenshot.jpg) of AviMux having finished (perhaps its of some use).
just to check, I passed the resulting file trough Virtual Dub once again (direct stream copy) an the resultimng file was same size as the original file (wich means 5MB more than the AVI-mux one), though the filesize was all it changed.:D

[alexnoe]

It was asynchronous? I thought I programmed to insert dropped frames instead of entirely leaving the chunk out :scared:

No idea why it runs out of sync, unless some index values of some audio chunks are screwed as well. If some audio chunks are also bad, then I'd have to add a more sophisticated rebuild method :angry:

[pomo]

It seems you're very interested in that file, in that case, I'll share it (FOR TESTING PURPOSE ONLY, AS IT IS SO "DESTROYED" IT IS NOT WATCHABLE, really I don't do this for illegal purposes, if you don't beleive me, read this thread).


I don't host the file anymore as my new computer makes to much noise
(as being a student, my computer stands 3 steps away from my bed ;) ). If there is still interest in that file I might find another host for it.

[pomo]

it's been some time since you last connected to my ftp so do you got the comlete file? And if that's the case, what do you think of it?

[alexnoe]

The file is strange...i will have to implement
reindexing to process it. => somewhen after the
last exam, i.e. after 24/02/03...

[Chibi Jasmin]

Originally posted by alexnoe
It was asynchronous? I thought I programmed to insert dropped frames instead of entirely leaving the chunk out :scared:



Just a question: In what case exactly does AviMuxGUI insert dropped frames (audio frames you mean I guess), when it encounters a corrupt audio frame? For CBR and/ or VBR MP3? As for AC3 I been begging for a long time, but maybe it's there for MP3?

[alexnoe]

MP3 streams which are broken at any location other than the beginning are rejected with a muxing error.

[Chibi Jasmin]

Originally posted by alexnoe
MP3 streams which are broken at any location other than the beginning are rejected with a muxing error.

Okay, thanx, I must have misunderstood that quote about inserting dropped frames instead of leaving the chunk out...

[MasterYoshidino]

file size increasing is likely to be caused by adding additional key frame information

the data rate can also be changed by veedub when you direct stream copy an avi with audio.

i.e. adding an mp3 stream to an avi that has no audio

time i tried downloading the file and it was offline so this is just an hypothesis

[alexnoe]

file size increasing is likely to be caused by adding additional key frame informationYou can't add additional keyframe information.