mplayer -dumpstream: issues, circumventions, solutions

[jggimi]

I am struggling with a vob obtained through mplayer's -dumpstream facility, as my normal rippers, dvdbackup and dvdstream, are unable to deal with this particular disc (RipGuard protected).

The main content is title # 16, a multiangle PGC with 2 audio and 3 subtitle tracks. I used:mplayer dvd://16 -dvdangle 2 -dumpstream -dumpfile rip.vobThis vob plays in perfect sync. I had hoped to demux A/V into ac3 and m2v files, then use mplex and dvdauthor in preparation for burning.

Unfortuately, the elemental streams do not sync up. A tcprobe of the first 30 MB of the .vob show these sync differences:[tcprobe] MPEG program stream (PS)
[tcprobe] summary for rip.vob, (*) = not default, 0 = not detected
import frame size: -g 720x480 [720x576] (*)
aspect ratio: 16:9 (*)
frame rate: -f 23.976 [25.000] frc=1 (*)
PTS=0.2814, frame_time=41 ms, bitrate=9800 kbps
audio track: -a 0 [0] -e 48000,16,2 [48000,16,2] -n 0x2000 [0x2000]
PTS=0.0919, bitrate=448 kbps
-D 4 --av_fine_ms 22 (frames & ms) [0] [0]
audio track: -a 1 [0] -e 48000,16,2 [48000,16,2] -n 0x2000 [0x2000]
PTS=0.0919, bitrate=448 kbps
-D 4 --av_fine_ms 22 (frames & ms) [0] [0]
detected (3) subtitle(s)
detected (6) presentation unit(s) (SCR reset)
According to tcprobe, the difference in start time between video and audio is 0.1895 seconds. Nope. After running mplex with 0 delay, the difference (through using mplayer's + and - keys to try to get close) in the a/v streams is around 1.7 or 1.8 seconds. I'm NOT close enough, as mplayer only jumps by +/100ms.

Note that it shows 6 presentation units when probing the first 30 MBs. If I use the default 1 MB, it shows 5 presentation units and all in sync. If I run tcdemux with -W, I get 9 presentation units througout the content.

With every new PSU, PTS/DTS gets reset, IIRC. This explains why elemental streams (needed for mplex/dvdauthor) don't sync.

Any suggestions for getting sync'ed elemental streams?

I do need to remux, the .vob produced by -dumpstream has no VOB Units, which mplex will provide, and the video stream needs to be run through tcrequant to fit a DVD-5.

[jggimi]

I should add I've tried extracting elemental streams a variety of ways: mplayer -dumpaudio/-dumpvideo, tcextract, and tcdemux|tcextract.

[jggimi]

I've given up, and am manually syncing. I spent two days futzing with mplayer, as one can't easily disable -autosync, and switched to vlc to set what looks like a reasonable, if not perfect sync during mplex.

[alec_robertson]

Have you tried posting (mplayer-users@mplayerhq.hu) or searching (news.gmane.org/gmane.comp.video.mplayer.user) the mplayer mailing list? They're pretty good at troubleshooting these matters...

[jggimi]

I've tried on #mplayer (freenet IRC), and have googled my brains out for days, and searched every mplayer and mencoder related newsgroup.

Its ugly, but I have the manual, experimental circumvention.

[jggimi]

While this might have been a 90 minute simplistic process with DVD Fab Decrypter and DVDShrink, the drive could not be attached to a Windows platform, I had need to do this from a *nix box without Wine.

Tools used:
mplayer
spunnmux/spumux (packaged with dvdauthor)
streamanalyze (packaged with streamdvd)
tcrequant (packaged with transcode)
mplex (packaged with mjpegtools)
dvdauthor
mkisofs (packaged with cdrtools)
growisofs (packaged with dvd+rw-tools)
If anyone ever wants the process, I can post my notes.

[danpos]

@jggimi

I think that a mini "HOWTO" would be great for all novice linuxers (as me for instance :D).

Regards,

[jggimi]

Ok. Here are my notes from this particular process. Let's not call it a "HOWTO" since it really isn't.

Disclaimer: Prior to using any of these programs, it is an absolute must to read their man pages, and perhaps peruse other available documentation. I have described some operands, briefly, and they should not be depended upon as a sole source of information. Some settling of contents may occur during shipping. Objects in mirror may be larger than they appear. All models over 18 years of age.

------------------------------------------------------------------------

Source: a DVD-9 which cannot be used with the standard ripping and transcoding applications, such as dvdbackup or streamdvd.

Process requirements: conducted entirely with Unix-like applications, as the dvd drive/burner is a peripheral of an OpenBSD platform, where wine emulation dates from 1999 and is effectively unusable.

Object: a main-content-only DVD-5 with the desired angle, soundtrack, and subtitle.

The main content, Title #16, is a two-angle PGC with 2 soundtrack and 3 subtitle streams.

--------

mplayer dvd://16 -dvdangle 2 -dumpstream -dumpfile rip.vob-dumpstream copies all audio and sub streams to the -dumpfile. Angle 2 is the version of the video wanted.
mplayer rip.vob -dumpvideo -dumpfile dirty.m2vthis extracts the video elementary stream -- as it stands, in this case, it requires splitting by mplex due to PTS resets and other problems, source unknown. Called "dirty" to differentiate it from the later "clean" elementary stream.mplayer rip.vob -dumpaudio -aid 129 -dumpfile dirty.ac3 matching soundtrack. -aid 129 specifies the 2nd soundtrack.mkdir subs; spuunmux -o subs/sub -s 1 rip.vobThis graps the subs in spu format -- which is an .xml file and one .png file for every title. They are placed in their own directory because there can be more than a thousand files involved. -s 1 selects the 2nd sub stream.mplex -f 8 -o clean%d.mpg dirty.m2v dirty.ac3Due to PTS issues with this particular content, I end up with 5 mpeg files from this procedure -- clean1.mpg thru clean5.mpg. 1-3 and 5 are effectively empty of a/v, the 4th file was selected for all further processing. -f 8 is for DVDs, it adds "VOB Units" -- VOBUs -- to the program stream, which are needed by authoring programs like dvdauthor.mplayer clean4.mpg -dumpvideo -dumpfile clean.m2v
mplayer clean4.mpg -dumpaudio -dumpfile clean.ac3this gives us clean elementary streams. -aid is not needed because there is only one audio track in this .mpg file.tcrequant -i clean.m2v -o small.m2v -f 1.173I did the requant math by cheating -- first authoring a dvd using the subs, audio, and unchanged clean video, and running streamanalyze against it to get the requant factor.mplex -f 8 -o synced.mpg -O -2350 small.m2v clean.ac3-O is audio delay or advance for sync. I found the -2350 by experimentation. I created a test mpg by copying the first several hundred MB of clean.mpg with dd, and used VLC for delay testing, since mplayer's -autosync could not be turned off.mplayer synced.mpg
vlc synced.mpgI tested sync with multiple players.mkdir vobs; spumux subs/sub.xml < synced.mpg | dvdauthor -o vobs -t -f -This muxed in the subs, and sent the complete 3 stream program to dvdauthor. That program built VIDEO_TS and AUDIO_TS folders, and VOB files for this title.dvdauthor -o vobs -TThis builds the root table of contents, for a menuless DVD that players can use.mplayer dvd://1 -dvd-device vobs -sid 0Test the dvd structure and turn subtitle stream 0 on.mkisofs -R -o dvd.iso vobsThis makes the .iso file.mplayer dvd.iso -sid 0
Test again.growisofs -dvd-compat -Z /dev/rcd0c=dvd.isoThis burns a dvd-r or dvd+r.ogle /dev/rcd0cThis tests the dvd-r/+r with another player.----------------------------

Speeding up the process:There are many intermediate files here which are used sequentially. These can either be piped, as done between spumux and dvdauthor, or, if file names are required (such as with mplayer's -dumpvideo and -dumpaudio demuxing) these can be files which are named pipes (mkfifo), saving both hard disk space and elapsed time. File structures that require seeks, such as the VOB sets created by dvdauthor, cannot be piped.

[danpos]

@jggimi

Whatever be called this detailed procedure that you did use, I've to say that it's very clear to me and I think that it will be very usefull for everyone. :)

Thanks for sharing your findings with us.

Regards,

[shevegen]

A shame that someone cant put together a nice little app that handles these things in a GUI too :)

[jggimi]

I have an update, which is why I changed the thread title.

I ran a second disc through this process. Same distributor, same protection schema, similar VTS structures.

This time, I had 2 mpeg files out of the first mplex run, with 6 files total. Obviously, some sort of post-encoding edit -- or perhaps a layer transition -- caused a PSU reset 2/3rds of the way through the main content.

In order to deal with this one, I had to split and resync the spumux subs. No tool existed, so I wrote an awk script to do al the timing resyncs in the 2nd program stream.

In order to do the resyncronizing, I needed accurate lengths. mplayer -identify is useless, but googling found a way to get an accurate length for each program segment. The math in the output produces the length in 1/100 seconds based on the last frame.mencoder clean4.mpg -ovc copy -nosound -o /dev/null \
-quiet 2>&1 | awk '/^Video stream/{print $10+$10/$12}'I use tcsh, so my pipe of both standard output and standard error is simpler (!&) and I may have mistyped sh's version here. :)

A few corrections:
mkisofs needs the "-dvd-video" option in order to produce a proper UDF/CD9660 bridged filesystem. If you forget it, you'll make a really nice coaster. Readable, and mountable. Just not usable.
I found that piping (via FIFOs) from mplayer produced unusable results. Seeks are needed. Mplex, too, needs to seek for both input and output files. The only safe piping can be done with spumux and dvdauthor.
An error with spumux (version 0.6.11) drove me nuts. For some reason, on the second segment, it placed the substream with id 32, which "hid" subs from players, tcprobe, etc. Even though "s -0" is the default with spumux, I had to explicitly use it.
Because someone would ask for it if I didn't include it, here's my awk script. It was quick-and-dirty, all it does is timing changes. If you use it as-is, you must edit the .xml files into a part1, part2, etc at the breakpoints described by each segment's length.# ensure we have an offset, if not, end with error message:

BEGIN {

if (offset==0) {
print "set offset in seconds!"
print "use awk -v offset=nnnn.nn"
exit
}

}

# any xml that doesn't start with "spu" gets printed as is:
$1 !~ /<spu/

# we want to change the timestamps, reducing them by the number of seconds
# in the offset.

/<spu image/ {

# there may or may not be an "image" tag with a spu entry, I don't know why spumux
# makes entries without images, but we'll use and correct them too.

# format of the tags is:
# start="hh:mm:ss.hh" or end="hh:mm:ss.hh"

if (substr($2,1,5)=="image") {
rawstart=$3
rawend=$4
}

else {
rawstart=$2
rawend=$3
}

# now that we have the timestamps, convert to seconds:

starthour=substr(rawstart,8,2)
startmin=substr(rawstart,11,2)
startsec=substr(rawstart,14,5)
endhour=substr(rawend,6,2)
endmin=substr(rawend,9,2)
endsec=substr(rawend,12,5)

startsecs=starthour*3600+startmin*60+startsec
endsecs=endhour*3600+endmin*60+endsec

# subtract the offset:

newstartsecs=startsecs-offset
newendsecs=endsecs-offset

# convert back to timestamp values:

newstarthour=int(newstartsecs/3600)
secsleft=newstartsecs-newstarthour*3600
newstartmin=int(secsleft/60)
newstartsec=secsleft-newstartmin*60
newendhour=int(newendsecs/3600)
secsleft=newendsecs-newendhour*3600
newendmin=int(secsleft/60)
newendsec=secsleft-newendmin*60

# make new timestamp fields:

newstart=sprintf("start=\"%02d:%02d:%02.2f\"", \
newstarthour,newstartmin,newstartsec)
newend=sprintf("end=\"%02d:%02d:%02.2f\"", \
newendhour,newendmin,newendsec)

# stick them in the appropriate output field:

if (substr($2,1,5)=="image") {
$3=newstart
$4=newend
}

else {
$2=newstart
$3=newend
}
#and output:

print
}