How to rip DVDs using "structural protections" in linux?

[xanas3712]

Ok, what I've got is a bunch of new Funimation anime DVDs, as I purchase stuff regularly from them. In the past all of my old dvd collection hasn't been an issue, I'm able to make backups and do things the way I like. I prefer to watch without having to insert a disc as I use a media pc and like having everything easily accessible from a 1 TB external (the added bonus is it saves my disc from the lovely West Texas dust monsters).

In any case, I just got Tsubasa Volume 10, School Rumble Season 2 Part 2, and Claymore Volume 2. For some reason they all have some odd problems in linux, which I'm certain has something to do with weird copy protections. I'll see if anyone can figure out what I've got from the signs, as I'm not sure after searching google how to figure this out.

1) The discs play, strangely enough, in any player capable of going to the menu. So kaffeine works, and vlc works in dvd menu mode, and mplayer even works when using dvdnav://, but if I try to use gmplayer dvd://1 then the player will sit there thinking real hard for minutes and minutes (with numerous read errors in dmesg appearing). If I attempt to use -dumpfile option to dump the dvd to disc, it does work, but it takes about 10 times longer to do than normal (several hours instead of 10 minutes or so).

2) This vob file that I do finally get has a bit of "black space" at the beginning, but the video otherwise seems good. The problem, however, is that the audio doesn't get ripped right. It's only half there. The subtitles come out fine and the video encodes seemingly fine.

3) To resolve this, I had used dvdfab in windows to re-rip the vob just to get the audio. When I muxed the audio/video together though, there was major sync issues. So I decided to start over and do the video from the same vob. This time everything worked out fine. I was able to do all the important encoding/etc. using avidemux in linux, but I had to rip using dvdfab (a program I don't like much, I can't figure out how to prevent it from splitting the vob file, and the interface is constrained, not to mention not being FOSS/etc.)

In any case I'm wondering if anyone knows if there is a way around this or an FOSS tool available that can circumvent the protection being used here. If not possible, I'd even appreciate knowing how to just identify what protection it is so I can discuss on more linux-specific sites.

Ultimately I'd like to find a way to get something that will work in my bash script which I use to encode my movies into an x264/mkv file in 1 step.

Even if this isn't possible, if there is a program that will work well within a virtualbox or wine environment, that would do better than what I've got so far. I hate having to reboot just to rip the dvd, and I don't keep my windows partition high on space because I don't use it much at all.

Ehhh.. well dvdfab does work in wine, lol. I swore I tried it yesterday and it didn't, but apparently I tried dvddecryptor instead, and it didn't work.

Well anyway, dvdfab works fine in wine, it's got a platinum rating and is working fine for me.

That said, it's still not an ideal solution because it can't be used in a script, unless there is a way to command line it that I can't find.

[RunningSkittle]

dvdfab has a CLI

[xanas3712]

Ok, so where is it documented? I can't find any instructions for anything but the GUI on the dvdfab website?

[JohnAStebbins]

It seems to me that you almost answered your own question.

mplayer dvdnav://<title> -dumpstream -dumpfile movie.mpg

This dumps the selected title to the file movie.mpg. movie.mpg is a program stream that any transcoder should handle fine.

[xanas3712]

Hmm, you know what I didn't try that with dvdnav, let me do that and see if it works.

Ok, that does seem to work, I tried the same thing with vlc transcoding and it didn't work, which is why I assumed this wouldn't work (I guess that's what I get for not trying it).

I need to test this file to see if it dumps everything though, because I think it may not dump all the subtitle & audio tracks, so I may have to do multiple dumps to get the streams I want...

Well well, works fine after only doing it once.

I attached my amateurish little script, I did the mplayer line you gave manually since my script uses existing files as drop ins in case I need to do something manually like this.

I'll probably add an option to use dvdnav instead whenever there are issues.


#!/bin/bash
# Version 1.35

encoder=mencoder
#encoder=vlc
#encoder=ffmpeg
#manually set crop here, comment out if using automatic crop detection
crop="crop=720:480:0:0"
vonull=""
#vonull=-vo null
#dvd device
dvd=/dev/dvd
#dvd device mountpoint
mountdvd=/media/cdrom0
#language priority for track1
language1=en
mkvlang1=eng # can't use mkvlang[en] apparently as bash only supports integer arrays :/
#language priority for track2
language2=ja
mkvlang2=jpn
#max# of audio tracks per language above
maxlanguage1=1
maxlanguage2=1

#video encoding settings
#us release of anime settings - high quality
#content="US Anime DVD"
filters1="pullup,softskip,harddup"
filters2=",hqdn3d=luma=8:chroma=8:luma_strength=7:chroma_strength=8:pre=0"
fps="24000/1001"
fpsrate="23.976023976"
vquality=18
wantsubs="yes"
#end us release of anime settings - high quality

# auto to select title automatically on dvds were a single title has the movie
# manual will make user select the title# regardless
# modeS is segment mode where user has to specify a number of chapter ranges
# as well as the chapters that go with those segments
mode="auto"
#modeS="on"
modeS="off"
#mode="manual"
#concatenate="on"
concatenate="off"
#end option variables

if [ "$1" = "-h" ] || [ -z "$1" ]; then
echo "-------------------------Ripmaster V 1.20---------------------------
Basic bash script for encoding video to x264 mkv using mencoder to make conversion
of video from dvd to x264 a one step process.
----------------------------Commands-------------------------------"
echo "-h = help"
echo "-r name angle# titlecount startitle subifo# = rips movie - edit top of the script to set basic options
---------------------------Description-----------------------------
Rips dvd movie to x264 mkv using up to 2 subs (currently) and 2 audio tracks
options set in this file
Currently set for $content Mode: $mode
-r without parameters specified afterward will prompt user for input.
"
fi
if [ "$1" = "-i" ]; then
mplayer -identify -frames 1 -dvd-device $dvd dvd:// | grep TITLE
fi
if [ "$1" = "-c" ]; then
gmplayer -vf cropdetect -dvd-device $dvd dvd://$2
fi
if [ "$1" = "-r" ]; then
if [ -n "$2" ]; then
outf=$2
else
read -p "What is the name of this dvd? " outf
fi
if [ -n "$3" ]; then
angle=$3
else
read -p "Which angle do you want (Check with mplayer -dvd-device /dev/dvd dvd://title# -dvdangle angle#)? " angle
fi
if [ -n "$4" ]; then
titlecount=$4
else
read -p "How many titles do you want on this dvd (usually 1)? " titlecount
fi
if [ "${titlecount}" -gt "1" ] || [ ${mode} = "manual" ]; then
if [ -n "$5" ]; then
starttitle=$5
else
read -p "What's the starting title of the set? " starttitle
fi
if [ -n "$6" ]; then
subifo=$6
else
read -p "Is there a specific IFO# for subtitles? (0 will use ifo# matching title#) " subifo
fi
if [ "${modeS}" = "on" ]; then
i=${starttitle}
echo -n "enter amount of segments, chapters in segment, and starting chapter for ${i} below:"
echo -n ""
while read segs[${i}]; do
read seglengths[${i}]
read segchapstarts[${i}]
if [ "${i}" -ge "$((${titlecount}+${starttitle}-1))" ]; then break; fi
((i++))
echo -n "enter amount of segments, chapters in segment, and starting chapter for ${i} below:"
echo -n ""
done
echo ${segs[@]}
echo ${seglengths[@]}
echo ${segchapstarts[@]}
fi
title=${starttitle}
fi
echo "mkdir ${outf}..."
mkdir ${outf}
echo "cd ${outf}..."
cd ${outf}
echo "mounting ${dvd}..."
mount ${dvd} 2>&-
if [ "${titlecount}" = "1" ] && [ ${mode} = "auto" ]; then
if [ "`ls ifofile 2>&-`" = "ifofile" ]; then
echo "Using existing titlefile..."
else
du ${mountdvd}/VIDEO_TS/VTS*.IFO > ifofile
fi
i=0
while read line; do
ifolength[${i}]=`echo ${line} | sed 's/\([0-9]*\).*/\1/'`
ifo[${i}]=`echo ${line} | sed 's/.*\///'`
((i++))
done < <(cat ifofile)
high=0;
for ((i=0;${i}<${#ifo[@]};i=${i}+1)); do
testlencur=${ifolength[${i}]/.*}
testlenhi=${ifolength[${high}]/.*}
if [ ${testlencur} -gt ${testlenhi} ]; then
high=${i};
fi
done
ifotocopy=${ifo[${high}]}
echo "copying IFO..."
cp ${mountdvd}/VIDEO_TS/${ifotocopy} ./ 2>&-
else
cp ${mountdvd}/VIDEO_TS/VTS*.IFO ./ 2>&-
fi
if [ "${titlecount}" = "1" ] && [ ${mode} = "auto" ]; then
if [ "`ls titlefile 2>&-`" = "titlefile" ]; then
echo "Using existing titlefile..."
else
mplayer dvd://1 -dvd-device ${dvd} -nosound -vo null -frames 1 \
-identify 2>&- | grep TITLE_\[0-9\].*_LENGTH | sed 's/ID_DVD_TITLE_//'\
> titlefile
fi
i=0
while read line; do
length[${i}]=`echo ${line} | sed 's/.*=//'`
title[${i}]=`echo ${line} | sed 's/\([0-9]*\).*/\1/'`
((i++))
done < <(cat titlefile)
high=0;
for ((i=0;${i}<${#title[@]};i=${i}+1)); do
testlencur=${length[${i}]/.*}
testlenhi=${length[${high}]/.*}
if [ ${testlencur} -gt ${testlenhi} ]; then
high=${i};
fi
done
title=${title[${high}]}
fi
# detecting these once should be fine, hopefully audio/sub tracks will remain
# the same throughout the titles!
if [ "`ls subfile 2>&-`" = "subfile" ]; then
echo "Using existing subfile..."
else
mplayer dvd://${title} -dvd-device ${dvd} -nosound -vo null -frames 1\
2>&- | grep language\: | grep subtitle |\
sed 's/.*\([0-9]\).*\([a-z][a-z]\)/\1 \2/'\
> subfile;
fi
i=0;
while read line; do
sid[${i}]=`echo ${line} | sed 's/\([0-9]\).*/\1/'`
slang[${i}]=`echo ${line} | sed 's/.*\([a-z][a-z]\)/\1/'`
((i++))
done < <(cat subfile)
if [ "`ls audiofile1 2>&-`" = "audiofile1" ]; then
echo "Using existing audiofile1..."
else
mplayer dvd://${title} -dvd-device ${dvd} -nosound -vo null -frames 1 2>&- |\
grep language\: | grep aid | grep ${language1} |\
sed 's/.*language: \([a-z][a-z]\).*\([0-9][0-9][0-9]\)./\2 \1/'\
> audiofile1
fi
i=0;
while read line; do
if [ ${i} -lt ${maxlanguage1} ]; then
aid[${i}]=`echo ${line} | sed 's/\([0-9][0-9][0-9]\).*/\1/'`
alang[${i}]=`echo ${line} | sed 's/.*\([a-z][a-z]\)/\1/'`
fi
((i++))
done < <(cat audiofile1)
if [ "`ls audiofile2 2>&-`" = "audiofile2" ]; then
echo "Using existing audiofile2..."
else
mplayer dvd://${title} -dvd-device ${dvd} -nosound -vo null -frames 1 2>&- |\
grep language\: | grep aid | grep ${language2} |\
sed 's/.*language: \([a-z][a-z]\).*\([0-9][0-9][0-9]\)./\2 \1/'\
> audiofile2
fi
j=0;
i=${maxlanguage1};
while read line; do
if [ ${j} -lt ${maxlanguage2} ]; then
aid[${i}]=`echo ${line} | sed 's/\([0-9][0-9][0-9]\).*/\1/'`
alang[${i}]=`echo ${line} | sed 's/.*\([a-z][a-z]\)/\1/'`
fi
((j++))
((i++))
done < <(cat audiofile2)
echo ${aid[@]}
echo ${alang[@]}
if [ "`ls cropfile 2>&-`" = "cropfile" ] || [ -z ${crop} ]; then
echo "Using existing crop..."
else
#changed to playing only end of the file as I can't seek all the way through some
#particular dvd that I had... hopefully should be good enough to get proper crop
mplayer dvd://${title} -dvd-device ${dvd} -nosound\
-sstep ${speedstep} -ss ${startcropdetect} ${vonull} -vf\
cropdetect 2>&- | grep "vf crop" | sed 's/.*vf //' | sed 's/).//' \
> cropfile
fi
if [ -z ${crop} ]; then
while read line; do
crop=${line}
done < <(cat cropfile)
fi
#last crop value should be reliable enough to use...
if [ "${modeS}" = "on" ]; then
actualtitlecount=${titlecount}
titlecount=1
#for((s=${starttitle};${s}<$((${starttitle}+${actualtitlecount}));s=${s}+1)); do
# titlecount=$((${titlecount}+${segs[${s}]}))
#done
actualtitle=${startitle}
title=1
fi
for ((t=0;$t<${titlecount};t=$t+1)); do
echo "Extracting from ${dvd} to ${outf} titlenum=${title} angle=${angle} with quality"
echo "qp level=${vquality}, using audio=${alang[@]}, subs=${slang[@]}, ${crop}"
echo "extracting chapter information from selected dvd title..."
if [ "`ls ${title}-${outf}.txt 2>&-`" = "${title}-${outf}.txt" ]; then
echo "using existing ${title}-${outf}.txt..."
else
if [ "${modeS}" = "off" ]; then
dvdxchap ${dvd} -t ${title} 2>&- > ${title}-${outf}.txt
fi
fi
echo "mplayer dumping ${title}-${outf}.vob..."
if [ "`ls ${title}-${outf}.vob 2>&-`" = "${title}-${outf}.vob" ]; then
echo "using existing ${title}-${outf}.vob..."
else
if [ "${modeS}" = "on" ]; then
for ((s=0;${s}<${#segs[@]};s=${s}+1)); do
currenttitle=$((${starttitle}+${s}))
currentlength=${seglengths[${currenttitle}]}
for((cur=0;${cur}<${segs[${currenttitle}]};cur=${cur}+1)); do
currentstart=$((${segchapstarts[${currenttitle}]}+$((${currentlength}*${cur}))))
currentend=$((${currentstart}+${currentlength}-1))
echo "mplayer dumping chapters ${currentstart}-${currentend} to ${title}-${outf}.vob"
mplayer -dvd-device ${dvd} dvd://${currenttitle} -chapter ${currentstart}-${currentend}\
-dvdangle ${angle} -dumpstream -dumpfile ${title}-${outf}.vob 2>&-
((title++))
done
done
else
mplayer -dvd-device ${dvd} dvd://${title}\
-dvdangle ${angle} -dumpstream -dumpfile ${title}-${outf}.vob 2>&-
fi
fi
((title++))
done
if [ "${titlecount}" -gt "1" ] || [ "${modeS}" = "on" ]; then
if [ "${modeS}" = "on" ]; then
title=1
titlecount=0
for((s=${starttitle};${s}<$((${starttitle}+${actualtitlecount}));s=${s}+1)); do
titlecount=$((${titlecount}+${segs[${s}]}))
done
else
title=${starttitle}
fi
else
((title--))
fi
if [ "${concatenate}" = "on" ]; then
echo -n "Enter the number of concatenations you made below"
echo ""
echo -n "after making concatenations into consecutive vobs"
echo ""
echo -n "using the titlename-1/titlename-2/etc. structure"
echo ""
echo -n "concatenate using - "
echo ""
echo -n "cat titlename-1 titlename-2 > temp"
echo ""
echo -n "mv temp titlename-1"
echo ""
read -p "How many concatenations did you make? " concat
# not using, just do the concatenation manually for this mess!
#c=0
# echo -n "Enter below the number of segments in concat#$c
#"
# while read concats[${c}]; do
# s=0
# echo -n "Enter below the filename of the segment#$s
# "
# while read
# if [ "$c" -ge "$(($concat-1))" ]; then break; fi
# ((c++))
# echo -n "Enter below the number of segments in concat#$c
# "
# done
titlecount=${concat}
fi
for ((t=0;$t<${titlecount};t=$t+1)); do
echo "extracting subs... ${title}-${slang[@]}"
if [ "`ls ${title}-${outf}.idx 2>&-`" = "${title}-${outf}.idx" ]; then
echo "using existing ${title}-${outf}.idx..."
else
# here we hope that title# corresponds to the ifo's! seems to be right on my test dvd
# not always correct from other tests so added a manual mode, most of the time though
if [ "${titlecount}" -gt "1" ] || [ ${mode} = "manual" ]; then
if [ "${subifo}" -ne "0" ]; then
temp=${subifo}
else
temp=${title}
fi
if [ "${temp}" -lt "10" ]; then
temp=0${temp}
else
temp=${temp}
fi
ifotocopy=VTS_${temp}_0.IFO;
fi
for ((i=0;${i}<${#sid[@]};i=${i}+1)); do
#if [ ${modeS} = "on" ]; then
# mencoder ${title}-${outf}.vob -vobsubout ${title}-${outf}-old -ifo ${ifotocopy}\
# -vobsuboutindex ${i} -sid ${i} -o /dev/null -nosound -ovc frameno 2>&-
#else ... mencoder seems faster & a little more flexible
tccat -i ${title}-${outf}.vob -L | tcextract -x ps1 -t vob -a\
0x2${sid[${i}]} 2>&- > ${title}-${outf}-${sid[${i}]}
subtitle2vobsub -o ${title}-${outf} -i ${ifotocopy} -a\
${sid[${i}]} < ${title}-${outf}-${sid[${i}]}
#fi
done
#x=0
# while read line; do
# lines[${x}]=`echo ${line}`
# timecode[${x}]=`echo ${line} | grep [0-9][0-9]:[0-9][0-9] | sed "s/.*\([0-9][0-9]\):\([0-9][0-9]\):\([0-9][0-9]\):\([0-9][0-9][0-9]\).*/\1\2\3\4/" | bc`
# y=$((x-1))
# if [ -z ${timecode[${x}]} ] || [ -z ${timecode[${y}]} ]; then
# echo ${lines[${x}]} >> ${title}-${outf}.idx
# else
# if [ "${timecode[${x}]}" -gt "${timecode[${y}]}" ]; then
# echo ${lines[${x}]} >> ${title}-${outf}.idx
# fi
# fi
# ((x++))
# done < <(cat ${title}-${outf}-old.idx)
# mv ${title}-${outf}-old.sub ${title}-${outf}.sub
fi
echo "dumping audio streams... ${alang[@]}"
for ((i=0;${i}<${#aid[@]};i=${i}+1)); do
if [ "`ls ${title}-${alang[${i}]}-${aid[${i}]}.ogg 2>&-`" = "${title}-${alang[${i}]}-${aid[${i}]}.ogg" ]; then
echo "using audio ${title}-${alang[${i}]}-${aid[${i}]}.ogg"
else
echo "mplayer audio extraction loop= ${i}"
echo "mplayer extracting aid= ${aid[${i}]}"
mplayer ${title}-${outf}.vob -aid ${aid[${i}]} -dumpaudio -dumpfile\
${title}-${alang[${i}]}-${aid[${i}]}.ac3
a52dec -o wav ${title}-${alang[${i}]}-${aid[${i}]}.ac3 | oggenc -q4 -o\
${title}-${alang[${i}]}-${aid[${i}]}.ogg -
fi
done
if [ "`ls ${title}-${outf}.264 2>&-`" = "${title}-${outf}.264" ]; then
echo "using existing ${title}-${outf}.264..."
else
echo "beginning video encode from ${title}-${outf}.vob to ${title}-${outf}.264..."
mencoder ${title}-${outf}.vob \
-vf ${filters1},${crop}${filters2}\
-oac copy\
-ofps ${fps} -ovc x264\
-x264encopts qp=${vquality}:subq=6:bframes=4:8x8dct:frameref=5:partitions=all:b_pyramid:weight_b:threads=auto\
-of rawvideo -o ${title}-${outf}.264;
fi
if [ "`ls ${title}-${outf}.mp4 2>&-`" = "${title}-${outf}.mp4" ]; then
echo "using existing ${title}-${outf}.mp4..."
else
echo "packing raw x264 ${title}-${outf}.264 into ${title}-${outf}.mp4"
#mp4creator -c ${title}-${outf}.264 -rate ${fpsrate} ${title}-${outf}.mp4 .. used to use this in gentoo, using mp4box in ubuntu
MP4Box -fps ${fpsrate} -add ${title}-${outf}.264 ${title}-${outf}.mp4
fi
echo "merging final matroska video container.... ${title}-${outf}.mkv..."
if [ "`ls ${title}-${outf}.mkv 2>&-`" = "${title}-${outf}.mkv" ]; then
echo "using existing ${title}-${outf}.mkv..."
else
audiostring=""
for ((i=0;${i}<${#aid[@]};i=${i}+1)); do
if [ "${alang[${i}]}" = "${language1}" ]; then
mkvlang=${mkvlang1}
else
mkvlang=${mkvlang2}
fi
audiostring="${audiostring}--language 0:${mkvlang} ${title}-${alang[${i}]}-${aid[${i}]}.ogg "
done
substring=""
if [ "${wantsubs}" = "yes" ]; then
for ((i=0;${i}<${#sid[@]};i=${i}+1)); do
if [ ${slang[${i}]} = "wn" ] || [ ${slang[${i}]} = "xx" ]; then
slang[${i}]="en"
fi
substring="${substring}--language ${i}:${slang[${i}]} "
done
substring="${substring} ${title}-${outf}.idx"
fi
if [ ${modeS} = "off" ]; then
chapterstring="--chapters ${title}-${outf}.txt"
else
chapterstring=""
fi
mkvmerge -o ${title}-${outf}.mkv ${chapterstring} ${title}-${outf}.mp4 ${audiostring} ${substring}
fi
((title++))
done
fi

[RunningSkittle]

Ok, so where is it documented? I can't find any instructions for anything but the GUI on the dvdfab website?

... readme.txt
6. Command Line Switches
___________________

(You can get a basic version of this list via 'DVDFab.exe /?')

/SRC "<Folder>"
Used to select the source folder.
Example: /SRC "D:\VIDEO_TS\"

/DEST "<Folder>"
Used to select the destination folder.
Example: /DEST "C:\My DVD\"

/START
Used to start the decryption process automatically
when the program has finished initialising.
Basically, it just presses the 'Start' button for you!

/CLOSE
Used to close the program when the decryption process has finished.
Basically, it just presses the 'Close' button for you!