Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th March 2007, 22:12   #1  |  Link
TheBashar
Registered User
 
TheBashar's Avatar
 
Join Date: Jan 2002
Posts: 112
VFR MKV Sync Problems

I'm having trouble with the audio getting out of sync in merged vfr mkv files. I put together a simple test case to illustrate this. Can someone more informed than I please clear up what I missing?

All the files to reproduce my test case can be downloaded from:
http://www.gigasize.com/get.php/481342/VFRSyncTest.7z

Basically it goes something like this:
  1. Have AVS file which generates blank video and constant tone
  2. Call to DeDup's DupeMC records the duplicate frame statistics
  3. Another AVS (almost same) adds Info() and uses DeDup to remove what would have been duplicate frames.
  4. DeDup outputs timecode file
  5. Another set of AVS files are almost identical but with different color and tone.
  6. I use MeGUI to encode the video to MKV
  7. I use MeGUI to encode the audio with Nero AAC
  8. I use MKVMergeGUI to merge each video MKV, audio MP4, and corresponding timecode file
  9. I use MKVMergeGUI to repeatedly merge the two clips together in an A-B-A-B-A-B pattern

Problem: When you watch the ABABAB.mkv, you see that the audio tone switches in time with the color changes. But as the video progresses (actually as the merge points are passed) the color changes lag farther and farther behind the audio.

Observations that I don't understand:
  • If I do the alternating in one video clip, the video dooesn't get out of sync; only with the merges.
  • When I watch the A (or B) clip, in MPC I see only 2 distinct frames, though there should be 4.
  • When I watch the merged ABABAB clip, I see 4 distinct frames per color, except the last one which shows only 2 distinct frames.

I could use the audio delay parameter in MKVMergeGUI to fix this if I could figure out some pattern to how much the audio is getting advanced (or chopped I think). While the pattern seems predictable with this simple test, I've had trouble with real video sometime getting a little out of sync and sometimes a lot.

In case you don't want to download the 84kB archive, here are the AVS files so you can see how simple this test should be:

a1.avs:
Code:
vid = BlankClip(length=75, width=320, height=240, pixel_type="YV12", fps=15.0, audio_rate=22050, stereo=false, color=color_firebrick)
aud = Tone(length=6, frequency=440, samplerate=22050, channels=1, type="sine")
CLIP = AudioDubEx(vid, aud)
Trim(CLIP, 0, -CLIP.framecount, true)
DupMC(log="a.dup")
a2.avs:
Code:
vid = BlankClip(length=75, width=320, height=240, pixel_type="YV12", fps=15.0, audio_rate=22050, stereo=false, color=color_firebrick)
aud = Tone(length=6, frequency=440, samplerate=22050, channels=1, type="sine")
CLIP = AudioDubEx(vid, aud)
Trim(CLIP, 0, -CLIP.framecount, true)
Info()
DeDup(log="a.dup", times="a.tmc", maxcopies=20, maxdrops=20, decwhich=0)
b1.avs:
Code:
vid = BlankClip(length=75, width=320, height=240, pixel_type="YV12", fps=15.0, audio_rate=22050, stereo=false, color=color_turquoise)
aud = Tone(length=6, frequency=660, samplerate=22050, channels=1, type="sine")
CLIP = AudioDubEx(vid, aud)
Trim(CLIP, 0, -CLIP.framecount, true)
DupMC(log="b.dup")
b2.avs:
Code:
vid = BlankClip(length=75, width=320, height=240, pixel_type="YV12", fps=15.0, audio_rate=22050, stereo=false, color=color_turquoise)
aud = Tone(length=6, frequency=660, samplerate=22050, channels=1, type="sine")
CLIP = AudioDubEx(vid, aud)
Trim(CLIP, 0, -CLIP.framecount, true)
Info()
DeDup(log="b.dup", times="b.tmc", maxcopies=20, maxdrops=20, decwhich=0)
And this is one of the timecode files (the other is identical):
Code:
# timecode format v2
0.000000
1333.333333
2666.666667
4000.000000
Finally a note about the archive. It contains all the files to recreate the test videos as well as the videos themselves. I just want to note that it contains a "ababab.tmc" file which is a timecode file _extracted_ from the ababab.mkv file, NOT used to create the video file.
Attached Files
File Type: 7z VFRSyncTest.7z (83.7 KB, 39 views)
TheBashar is offline   Reply With Quote
Old 7th March 2007, 04:50   #2  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,565
I suspected when I read, but I confirmed when I made a test.

a is exactly 5000 ms long, and the last frame is 4966. But in the timecodes, note that the first frame b starts at 5333 ms, not 5000. This is a failure of the timecode v2 format: There is no duration for the last frame. You end up with mkvmerge having to make up a number - and it picks the length of the frame prior to the last. (That's just a guess, I'm still investigating the exact method.) Doing anything like this, you'll basically have to be absolutely sure you have two of the same length frames on the end.

Tacking the start time of the next segment onto the timecode file doesn't work, sadly. Perhaps Mosu can be convinced to support that.
foxyshadis is offline   Reply With Quote
Old 7th March 2007, 10:34   #3  |  Link
TheBashar
Registered User
 
TheBashar's Avatar
 
Join Date: Jan 2002
Posts: 112
Some More Clues and Confusion

Quote:
Originally Posted by foxyshadis View Post
a is exactly 5000 ms long, and the last frame is 4966. But in the timecodes, note that the first frame b starts at 5333 ms, not 5000.
Thanks for looking into the problem foxyshadis. Where did you get the 4966 number from? I don't know how you were able to dig that out.

I've done some investigating of my own, and continue to be perplexed by what I find. I re-mkvmerged the individual a.mkv and b.mkv with artificial timecode files. I constructed these so each successive frame's duration is 100ms more than the previous.

[Note: I added a DeDup override to keep the first frame so you'll see an extra frame from my previous post's tests - now 5 frames per clip]

inca.tmc: (pseudo syntax to save space here)
Code:
0.000000, 100, 300, 600, 1000
incb.tmc: (pseduo syntax to save space here)
Code:
0.000000, 500, 1100, 1800, 2600
I then mkvmerge these two timecoded videos together (ababab) and extract the merged video's timecode file:

mergeinc.tmc: (pseduo syntax etc etc)
Code:
0.000000, 100.000000, 300.000000, 600.000000, 1000.000000,
1100.000000, 1600.000000, 2200.000000, 2900.000000, 3700.000000, 
4200.000000, 4300.000000, 4500.000000, 4800.000000, 5200.000000,
5300.000000, 5800.000000, 6400.000000, 7100.000000, 7900.000000,
8400.000000, 8500.000000, 8700.000000, 9000.000000, 9400.000000,
9500.000000, 10000.000000, 10600.000000, 11300.000000, 12100.000000
Notice here, at each merge point the duration of the previous clips last frame is set to the duration of said clips first frame. This would be an understandable rule IF it held true!

Here's the kicker, I got the "real" a.tmc file and I manually changed the duration of the first frame so if I add that duration to the last frame it would be exactly 5000ms - what the clip length is. So here's my timecode file:

a.tmc:
Code:
0.000000, 933.333333, 1400.000000, 2733.333333, 4066.666667
933.333333 + 4066.666667 = 5000 right?

b.tmc:
Code:
0.000000, 66.666667, 1400.000000, 2733.333333, 
4066.666667
Now, after mkmerging these into the individual video files, mkvmerging the two videos together, and extracting the merged video's timecode file I, unfathomably find:

merged.tmc:
Code:
0.000000, 933.000000, 1400.000000, 2733.000000, 4067.000000,
5400.000000, 5467.000000, 6800.000000, 8133.000000, 9467.000000,
10801.000000, 11734.000000, 12201.000000, 13534.000000,  14868.000000,
16201.000000, 16268.000000, 17601.000000, 18934.000000, 20268.000000,
21601.000000, 22534.000000, 23001.000000, 24334.000000, 25668.000000,
27002.000000, 27069.000000, 28402.000000, 29735.000000, 31069.000000
Now mysteriously the merge points all have durations of 1333ms - which is a totally different pattern than the previous test. Is it now looking at the duration of the 2nd to last frame?

IF I could figure out how it was determining the inter-merge duration, I could write a utility to patch the timecode file(s) to make it fit the actual video length. But, even if I could figure that out, my trouble doesn't end there.

I made an artificial timecode file which I knew would match the actual merged ababab and used mkvmerge to replace the timecodes on the merged video:

replace.tmc:
Code:
0.000000, 1000.000000, 2000.000000, 3000.000000, 4000.000000
5000.000000, 6000.000000, 7000.000000, 8000.000000, 9000.000000
10000.000000, 11000.000000, 12000.000000, 13000.000000
14000.000000, etc, etc, etc
Now, when I view this new video, the audio slowly falls behind the video at each successive merge point! Now I now the timecodes for the video match the exact clip lengths. What is causing the audio to be elongated or delayed?

Current Conclusions:
  1. Audio sync problems are caused (in part) by mkvmerge assigning final frame durations based on some algorithm other than looking at audio length.
  2. Audio sync problems are also caused by a yet unknown secondary source related to the merge points.

Though you still need some of the files from the link in the first post, you can get the updated scripts and files to reproduce these tests at:
http://www.gigasize.com/get.php/4839...dated_Files.7z
Attached Files
File Type: 7z Sync Test Updated Files.7z (84.5 KB, 31 views)

Last edited by TheBashar; 7th March 2007 at 19:26. Reason: Added Updated Files To Reproduce
TheBashar is offline   Reply With Quote
Old 7th March 2007, 13:15   #4  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,565
Bizarre. (I just meant the last frame pre-decimation with dedup, sorry.)

Since the behavior is unpredictable, the only way to work with it will be to fix mkvmerge. Adding the capability of reading a final end-duration timecode would probably be the most correct solution. I'll point Mosu to the thread. Once that's done, it'll be much simpler to find out if there are any further audio merge problems.
foxyshadis is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.