Bug: Progressively out-of-sync audio/video in DivX and Xvid files made by AutoGK

[pcjr]

YES! I can post finally!

Hi, I'm using AutoGK to transcode MPEG-2 files to Xvid or DivX. I notice that the transcoded audio and video becomes progressively more out of sync during playback. This is happening in both stable versions AutoGK 1.6 and 1.95. The sync problem of the DivX/Xvid files occurs in playback in both Windows and in Mac OS X (both QuickTime and VLC). The original MPEG-2 files (created by Snapstream Beyond TV) have no such sync problems. Therefore, there appears to be a bug in AutoGK during the transcoding process.

The sync problem used to not happen before. So what's changed in my computer setup? Before I was using my ATI All-in-Wonder to capture TV with Snapstream's Beyond TV, using software MPEG encoding. Because it's software encoding, I sit the bitrate to only 5224 kbps. Then I got a Hauppauge WinTV PVR2 that allows for hardware MPEG encoding. I set the bitrate to 5974 kbps, slightly higher. Nothing else was changed, as far as I know. Maybe AutoGK cannot handle the slightly higher bitrate? Maybe the MPEG encoding algorithm of software vs. hardware is different, and AutoGK can't handle the Hauppauge's version?

# Your computer config. (i.e. Media Players, Hardware (specifically: graphics card, Operating System, CPU) )
ATI All-in-Wonder 9000 card, Hauppage WinTV-PVR2 external USB2 MPEG hardware encoder, Windows XP professional, Pentium 4 2.4 Ghz, 1 GB RAM, Maxtor ATA133 drives (60GB and 250GB).

[pcjr]

So I changed my MPEG encoding bitrate to 5224 kbps, but still made the encoding via the hardware MPEG encoder. Looks like I still get progressively out-of-sync audio and video of the DivX/Xvid files.

I will change my TV capture recording to software and see if the problem with AutoGK is with its ability to use hardware-encoded MPEG source files.

[jggimi]

Hello, and welcome to the forum.

I'm moving this to the capture forum, where I think you'll get a more focused audience.

I know that you've got what appears to be "good" .mpg files -- in that your a/v is in sync.

Just to check, try demuxing the video and soundtracks to separate files. (You can use TMPG, DGIndex, or VdubMod, etc.) After demuxing, what is the length of your separate audio file? Does it match the length of the video?

[pcjr]

Hi, looks like in the DGIndex, the audio timestamp length was 31:49 as well as the actual length of the video -- the same. So, I don't think it's necessarily a capture problem but just somehow something wrong in how AutoGK handles the hardware-encoded MPEG files. I'll switch to software-encode (which I was using before the sync problem arose) and see if that fixes the problem.

[jggimi]

It sounds like you just examined the .d2v. If so, you may still have a problem. Make sure you demux the .m2v and the audio track into separate files, and only afterwards, compare the lengths of the video file and the audio file.

[stephanV]

are you by any chance using CBR MP3 (and if so, what bitrate and sampling rate)?

(do what jggimi said first though)

[pcjr]

jggimi, can you outline the steps on how to demux in DGIndex (steps as in menu listings, etc.)? If I only looked at the .d2v, then maybe you can let me know how to do it differently.

Also, stephanV, looks like I have variable bitrate encoding unchecked, so I guess it means CBR. Avg video bitrate at 5000 or 5750 kbps at 29.97 fps.

--Peter

P.S. I looked at the AVIs converted from software-MPEG encoded files and now it looks like I'm getting the same problem as well. So I don't know what's going on really.

[eb]

@pcjr,

Try to use DVD2AVI or DVD2AVIT3 and to get speed (and quality also) use ffdshow video codec, Resulted video and audio files mux in VirtualDubMod but use data about audio DELAY from audio header to set delay in INTERLEAVing and here you can set to convert to .mp3 but avoid lame rather.
Check also in VIDEO > FRAME RATE if there is some problem with audio length against video if such set to the framerate as prompted but few lines below set again to required framerate eg.25.000


eb

[pcjr]

eb, I have no idea what you're talking about. I'm just using AutoGK to convert mpg files automatically. I'm trying to do things automated so I'm not going to do the stuff below. Thanks for the suggestions though!

--Peter


@pcjr,

Try to use DVD2AVI or DVD2AVIT3 and to get speed (and quality also) use ffdshow video codec, Resulted video and audio files mux in VirtualDubMod but use data about audio DELAY from audio header to set delay in INTERLEAVing and here you can set to convert to .mp3 but avoid lame rather.
Check also in VIDEO > FRAME RATE if there is some problem with audio length against video if such set to the framerate as prompted but few lines below set again to required framerate eg.25.000


eb

[eb]

I hope that my information is useful for others.

eb

[pcjr]

Well, I'm a total newbie, so I don't know much about converting mpegs to avis using other programs. I just use autogk, and with the sync problems I've been getting I just want to determine whether autogk has a bug in it or not. I'm now pretty sure autogk is really buggy, because it seems a bit strange that AVIs made from both my software and hardware encoded mpegs have the same sync problems, and it would seem very unlikely that separate encoding methods of source files would lead to the same problem. The only thing I can think of is that this problem must have evolved from autogk during a version update. Of course I would also like to rule out problems with the mpeg source files, but again I'm a newbie so if someone could point out the menu details in how to demux in DGIndex for me to compare audio and video lengths that would be great.

[jggimi]

jggimi, can you outline the steps on how to demux in DGIndex (steps as in menu listings, etc.)? If I only looked at the .d2v, then maybe you can let me know how to do it differently. Open your MPEG-2 file (File...Open)

Set Audio to demux your audio track if it isn't Linear PCM (Audio...Output Method...Demux). If it is Linear PCM (.wav), then use Audio...Output Method...Decode to WAV instead.

Select the appropriate Audio track (Audio...Track Number)

Demux both audio and video (File...Save Project and demux video)The separate video (<name>.demuxed.m2v) and audio (<name>.<type>) will be in your folder. Examine the lengths of these files.

[pcjr]

So after demuxing, and opening the demuxed video in DGIndex, it's indeed 31:58, and in Windows Media player, the audio file is also 31:58. So it looks like the MPEG file is fine. It would be great if someone can accept a test MPEG file that my computer and Beyond TV software actually created, to see if someone else gets the same sync problems when they use AutoGK. Cuz with intact MPEG files, it all points to AutoGK being buggy in converting to AVIs. Anyone else have this problem, or is it just me?

[jggimi]

I'll accept a test file -- expect a PM from me shortly.

[jggimi]

I've received a 38MB fragment -- it contains only a broadcast advertisement with voice over, so its hard to tell if there's a sync problem. I did convert it to a 25MB XviD -- its "test.avi" in the /pub directory of my FTP server. I used AutoGK, my log is below.[6/1/05 4:05:13 PM] AutoGK 2.08b
[6/1/05 4:05:13 PM] OS: Win98SE (4.10.67766446).1
[6/1/05 4:05:13 PM] Job started.
[6/1/05 4:05:13 PM] Input file: test.mpg
[6/1/05 4:05:13 PM] Output file: C:\z\test.avi
[6/1/05 4:05:13 PM] Output codec: XviD
[6/1/05 4:05:13 PM] Audio1: Audio Stream 0 MPEG
[6/1/05 4:05:13 PM] Subtitles: none
[6/1/05 4:05:13 PM] Format: .AVI
[6/1/05 4:05:13 PM] Target quality: 75%
[6/1/05 4:05:13 PM] Started encoding.
[6/1/05 4:05:13 PM] Demuxing and indexing.
[6/1/05 4:05:15 PM] Processing file: C:\z\test.mpg
[6/1/05 4:05:15 PM] Source aspect ratio: 4:3
[6/1/05 4:05:15 PM] Source resolution: 720x480
[6/1/05 4:05:15 PM] Found NTSC source.
[6/1/05 4:05:15 PM] Analyzing source.
[6/1/05 4:06:24 PM] Source has percentage of interlacing in motion areas: 34.36
[6/1/05 4:06:24 PM] Source has percentage of telecined patterns: 67.16
[6/1/05 4:06:24 PM] Source has percentage of progressive patterns: 21.57
[6/1/05 4:06:24 PM] Source has percentage of interlaced patterns: 11.27
[6/1/05 4:06:24 PM] Source is considered to be hybrid (mostly FILM).
[6/1/05 4:06:24 PM] Looking for optimal hybrid thresholds.
[6/1/05 4:06:36 PM] Found threshold of: 1.83
[6/1/05 4:06:36 PM] Output will contain 1196 frames
[6/1/05 4:06:36 PM] Encoding audio.
[6/1/05 4:06:40 PM] Running single pass encoding.
[6/1/05 4:06:40 PM] Writing the following script to C:\z\agk_tmp\test_movie.avs
===========================================================
LoadPlugin("C:\PROGRA~1\AUTOGK\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\AUTOGK\filters\autocrop.dll")
LoadPlugin("C:\PROGRA~1\AUTOGK\filters\decomb.dll")

movie = mpeg2source("C:\z\agk_tmp\test.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
fixed_aspect = 0.888888888888889
c_width = width(cropclip)
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
out_width = 720
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
Telecide(movie,order=1,guide=1,post=2).Decimate(mode=3,threshold=1.83)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
LanczosResize(out_width,out_height)
===========================================================
[6/1/05 4:08:51 PM] Duration was: 2 minutes 10 seconds
[6/1/05 4:08:51 PM] Speed was: 9.16 fps.
[6/1/05 4:08:51 PM] Job finished. Total time: 3 minutes 37 seconds

[pcjr]

I'll resend. For some reason it got interrupted.

[jggimi]

I got the complete 1.5 GB file.

I've finished my analysis, and I have determined what the problem is, and what you can do to fix it. As I'd suspected, your video and audio are not the same length. I first opened the .mpg file you uploaded to my server with VirtualDubMod. The video is 29.97fps, and contains 57226 frames (31:49.443). The audio is MP2, and has a length of 31:52.248.

Its longer than the video, by 2.805 seconds. I then demuxed the audio and video with DGIndex. DGIndex popped up: "WARNING! Opening GOP is not closed. The first few frames may not be decoded correctly." A GOP is an MPEG term, and stands for Group-of-Pictures. They start with a Keyframe (I-frame), and then may have other frame types. This warning message pops up if the first GOP is corrupted, which can happen when capturing with MPEG.

The demuxed .m2v and .mpa track lengths matched VdubMod's numbers.What can you do about it? Adjust the length of the audio to match the video.

You can use BeSweet and one of its GUIs (BeSweetGUI, BeLight) to adjust the audio length. I recommend outputting the audio in WAV format, so that you are not using a lossy codec in this step.

I also recommend then using Cuttermaran; its an MPEG-2 editor that you can use with .m2v input for video, and .wav input for audio. You'll be able to set your start and end points and even cut out advertising. It includes a built-in muxer so that you can produce a new .mpg for use by AutoGK. If you make your cuts on I-frames (keyframes), you will not need to do any MPEG encoding with Cuttermaran.

I'll do this with the sample you sent me, and send you a PM when the .avi is available from my ftp server for download.

[pcjr]

Thanks for your help on this. Very strange indeed. I wonder why DGIndex reported a video length of 31:58 and Windows Media player reported an audio length of 31:58 as well. Are they reading from somewhere else in the file without actually checking the lengths?
Also, why can mpeg files be played with audio and video in sync by any player even though they're different lengths, but when converted to AVI the different lengths become apparent?

[pcjr]

Hi, I wrote to Snapstream about the MPEGs being created and this is what they say about their product, Beyond TV:

"The hardware encoder gives us a file and we cannot control the length of the video or audio. A compressed MPEG-2 file is delivered to us for writing on disk.

Since you are having this issue with both the hardware and software cards it seems the issue you are having isn't releated to our software but with the conversion software."

Beyond TV just gets the MPEG file from a hardware encoder but also has a built-in software encoder if the user has no hardware encoder. They have a point though -- since my AVI files are out-of-sync whether the source file came from a software-encoded or hardware-encoded MPEG, even if the file lengths were different, isn't this perhaps some standard in MPEG files that AutoGK should know how to handle? I used to use an older version of Beyond TV that had an MPEG-to-DivX converter plug-in that didn't have sync problems. (Unfortunately this was removed due to licensing issues, hence my turning to AutoGK.)

[len0x]

few points:
- different length of video and audio indicates problems in mpeg stream (dropped frames, garbage packets etc). But its not noticable since the format is designed to be streamable and therefore recover from any errors. Avi format is not streamable and during playback dshow filter is adjusting length of audio to match video (that is why asynch occurs)
- you have to process mpeg file with tools like ProjectX to correct mpeg stream and then use other tools like AutoGK which are processing video and audio separately.

[eb]

- different length of video and audio indicates problems in mpeg stream (dropped frames, garbage packets etc) Yes, this is true.

But its not noticable since the format is designed to be streamable and therefore recover from any errors. As long as droputs are not too big it is not noticable, and as long as time stamps are kept as delivered by broadcaster there is no desynchro symptoms.
Even if dropouts are big there is no problems to keep synchroinization, /piszczit,treszczit da lieziet/
In this particular case, described above in thread, AutoGK is NOT GUILTY.
Guilty is program that rewrites audio timestamps, so after corrections because of audio delay, later timestamps should be kept in stricte corelations to their video partners/frames.

/piszczit,treszczit da lieziet/
Kind ask to Russian fellows to translate this to English.

eb

Greetings to lenOx.

[jggimi]

There are other options -- I'd already mentioned adjusting audio length to match video. Pcjr could go the other way, and adjust video framerate to match audio, using VdubMod. But that would preclude using AutoGK.

I've been playing with the Soundtouch library to adjust speed and tempo on the sample pcjr sent me. Even though that was my earlier recommendation, I've not had much success with syncing things up. I've used BeSweet (BeLight/BeSweetGUI) as well as Audacity, and I've not been happy with the results. I'm now experimenting with adjusting video framerate (along with IVTC)... That may be a solution -- I'm not familiar with ProjectX, but it sounds like that may do the trick for preprocess correcting rather than going the Demux/Adjust/Remux route I've tried, or the framerate adjustment in VdubMod I'm running right now.

Pcjr, are you planning on using your MPEG-4 output with a standalone player or with a PC only? If PC, then perhaps adjusting video framerate may be a simple solution (though you'd need to learn to use VdubMod).

[jggimi]

And regarding ProjectX, which I've never used ... Doom9 has two guides:

http://www.doom9.org/DigiTV/projectx.htm
http://www.doom9.org/DigiTV/projectx-fullguide.htm

[eb]

Is synchronization OK when playing sample?

[jggimi]

The sample .mpg? yes. But obviously, not once demuxed/remuxed.

My problem with Soundtouch is one of inaccuracy. Using "soundtouch(-r ...)" changes I can get close, but not close enough by the end of his 32 minute sample.. Plus there seem to be bbMPEG mux problems when I transcoded mp2->mp2 with tempo change, which I circumvented this by making the change during an mp2->wav transcode, then just doing a simple wav->mp2 for muxing.

This seems way too complex a process for such a simple adjustment.

But when I shifted framerate rather than sound, it seemed to match up much better. I've got a version (at 23.94fps) that I'll put in my /pub directory right now for pcjr to look at -- well, listen to, since its a 320x240 at Q20, a really lousy encode but it kept it under 300mb. Its mostly to determine if the odd framerate is usable for him.

(Unfortunately, pcjr's sample is dubbed anime; I've been using the advertisements to check lip sync.)

Pcjr, let me know when you've got the thing, and whether the framerate would be acceptable -- ignore the video quality.

[gircobain]

I've used Audacity before for stretching/reducing non-matching audio tracks for cartoons
I've found it to be pretty accurate
The results were satisfactory IMHO

[jggimi]

Well, the difference between the soundtrack and the video is 0.146901%, kind of difficult for me to set with Audacity. I can set a new length in seconds (with change tempo) but ... it seems to fixate on 0.1% or 0.2%, and doesn't give the proper length.

With BeSweet, I can use a "framerate" alteration -- the soundtouch(-r ...) operand -- but I'm limited to relative video framerate values in milliseconds. The closest I can get is soundtouch (-r 29926 29970), and that did not work well for me. But, if I went the other way, and adjusted the video framerate to 29.926 instead, it seemed to match up ... well enough for dubbed anime, anway.

[len0x]

Another option can be to find a program that demuxes audio correctly. In my experience mpeg-vcr and mplayer do good job in some cases. (audio has to be remuxed to the final avi obviously after manual demuxing and encoding).

[pcjr]

Hi, guys, thanks for your help on this. It looks like because of MPEG capture issues like dropped frames, I would have to manually convert my MPEGs to AVIs. Though it would be great if I could use AutoGK to do this. I got this information from the Snapstream Beyond TV developer:

"I talked to a developer and he informed me that this issue is common to MPEG. The way to properly handle this is by using timestamps when muxing audio and video.

Since the hardware encoder has the same issue as the software encoder this seems to be an issue that is not under our control."

Is there a way to add timestamp handling by AutoGK?
I'm downloading the file now, jggimi. As for my use, I'm using it on my PC but would like to access Xvid content from a D-link DSM-320 and perhaps play it in a DVD player that recognizes DivX.

[pcjr]

Jggimi, so it looks like the AVI file you made has very good sync, especially when compared to MPEG files being dumped straight to AutoGK where I get a second or so desync by the end of the video file. Just wondering though how AutoGK handles audio and video. Are the timestamps from the original MPEG being followed? The people at Snapstream suggested that using the timestamps would help.

[pcjr]

Hey, not sure what happened, but I recently updated AutoGK to v2.09beta (I believe I had 2.08 previously as well as the stable version 1.95), and it looks like the most recent AutoGK Xvid file's audio and video are in sync, even towards the end of the file. Care to explain what feature was changed in the latest AutoGK, lenox? :thanks: The only new issue I see is that once in a while the picture gets all blocky but I'd rather have this than the desyncs. It turns out that the blocky areas are where the original MPEG seems to have dropped frames, so it looks like the Xvid file is even better in that it at least restores parts of these scenes upon playback. I'll continue to monitor future AutoGK autoconversions to see if the syncing holds up.

[jggimi]

...it looks like the AVI file you made has very good sync...

Glad to know it. Remember, though, that the sample has an unusual framerate, and is very unlikely to work with a hardware player and television.

...Care to explain what feature was changed in the latest AutoGK, lenox?...

Len0x has already published this information. If you would taken a moment to examine the change log yourself (packaged with AGK and published in the first post in the utterly humungous base AutoGK thread (http://forum.doom9.org/showthread.php?t=64266)) you'd see:VERSION HISTORY
2.09 beta

- ess compatibility option is working again (broken in 2.08)
- packed bitstream option of XviD is off again (make sure to reinstall packaged XviD)
- input file analysis is improved to detect more audio streams with larger delay
- same audio track from single track input source can be used for both first and second tracks again
.
.
.

[pcjr]

Yes, I definitely noticed the release notes but not sure which improvement in the latest release was responsible for the sync problem to be resolved or if it was just some tweak that seemed trivial but in fact did a whole lot more. My best guess was that it might have to do with detecting larger audio delays but I thought the problem was more of dropped frames inherent in many MPEG captures and that timestamps were involved?

[pcjr]

Hey, it looks like I spoke too soon. So the one time I got the audio and video in sync from AutoGK was apparently a one-time deal. A more recent recording shows the audio and video from the converted MPEG to be out of sync once again. I'm not sure why I got audio and video in sync that one time, except for the fact that I notice occasionally-to-often, the Xvid or DivX file size varies greatly. Even though I specify the size to be 33% of the original (for a half-hour show, the original is about 1.4 GB), sometimes the Xvid file is under 300 MB, sometimes it's over 300 MB, and one time it did something truly bizarre and produced an 8 GB file size! So I'm guessing this is somehow related to the audio/video sync issue.

[jggimi]

The type of encoding you are doing is a single pass, quality-based encode. File size will always be unpredictable. The only way to obtain a predictable size is with a multipass encoding.

To learn more about MPEG-4 encoding techniques, I recommend the DivX User Guide: http://www.divx.com/divx/divxpro/guides/. Yes, there are some DivX-specifics in the guide, but for the most part, the discussion is applicable to all MPEG-4 ASP codecs.

AFAIK, there are no "timestamps" in an AVI container for sync correction. As mentioned, its a simpler container.

I don't know your hardware, but if you use your soundcard for audio capture, and if your have (or can use) a WDM or VfW compatible video capture driver, you might consider capturing directly into an AVI container in HuffYUV or MJPEG, rather than continuing to use MPEG2. If that's the case, you can follow the recommendations and processes in our capture guide: http://www.doom9.org/capture/start.html.