first attempt at cmd line iso decrypter based on libcdio/udf and dvdcss

[qyot27]

Speaking of libdvdread, I ran into a roadblock on it when trying to get it compiled, because of 'libdl' and a 'dlopen' call not being present. I couldn't go any further than that, which was why I had to settle on using VLC for the drive authentication instead of continuing to try building lsdvd. A Google search seemed to point to those being a part of libc6 that handles dynamic linking, so I would guess the Handbrake patch for large file support wouldn't matter on that point?

And yeah, the ISO I was testing earlier was from a dual layer, a full 7.47GB. I'm not sure if I have any DVDs with content under 4GB that have encryption on them. The ones I know of under that size are discs I've authored or manipulated myself, and thus would lack encryption from the start.

EDIT: Where exactly should the spotter_open call be inserted in libdvdcss' source?

[spotter]

hack libdvdcss to get it to work (working right now for me)

what I did. added to src/device.c the function I wrote before the libc_open() function

then in src/dvdcss/dvdcss.h added a prototype for it before the dvdcss_open() prototype.

then edit my program's call from open(file, open mode) to spotter_open(file)

and running now on a dvd of mine in windows with DVDCSS_VERBOSE=2

[qyot27]

What's the actual prototype line to add in dvdcss.h? I have zero programming experience to know that sort of thing intuitively. The previous times I mentioned things like that were mostly by context of the references to it I could find from a search, but I can't see the context on that to know what it should look like (aside from copying the dvdcss_open prototype and replacing the name).

And I'm guessing open(file, open mode) is the if ((fd = open(argv[1], flags)) < 0) { line? So should that be

if ((fd = spotter_open(argv[1], flags)) < 0) {
or
if ((fd = spotter_open(argv[1])) < 0) {
?

[spotter]

what I added (and some context to see where I added it). I'll note, this is a total hack. don't learn anything from this approach. :)

src/device.c


/* Following functions are local */

int spotter_open(char const * file)
{
return open(file, O_RDWR | O_BINARY);
}

/*****************************************************************************
* Open commands.
*****************************************************************************/
static int libc_open ( dvdcss_t dvdcss, char const *psz_device )
{


src/dvdcss/dvdcss.h


LIBDVDCSS_EXPORT dvdcss_t dvdcss_open ( char *psz_target );
LIBDVDCSS_EXPORT int spotter_open(char const * file);
LIBDVDCSS_EXPORT int dvdcss_close ( dvdcss_t );


my code


printf("argv1 = %s\n", argv[1]);
if ((fd = spotter_open(argv[1])) < 0) {
perror("failed to open input/output file");
return 1;
}

[qyot27]

I recompiled libdvdcss and the decrypter and it seems to be working now, it recognized the ISO and there's definitely work occurring, as indicated by the orange light on my tower. We'll see how it turns out.

EDIT: The task finished, and MPC now detects no scrambling. It worked. Although the previous point about it not being a troublesome disc in the first place is still true.

[spotter]

Playing around w/ imaging via imgburn. it complains about it being a scrambled disc (which it is), unsure if it does authentication or not, but authenticated via vlc. imgburn made a scrambled disc (like ddrescue) + an MDS file and my utility descrambled it. Not a troublesome disc with bad sectors.

[spotter]

I recompiled libdvdcss and the decrypter and it seems to be working now, it recognized the ISO and there's definitely work occurring, as indicated by the orange light on my tower. We'll see how it turns out.

EDIT: The task finished, and MPC now detects no scrambling. It worked. Although the previous point about it not being a troublesome disc in the first place is still true.

my experience says you won't have any different experience w/ more troublesom discs.

The main way to attack this would be to confuse my "naive" method of seeing where VOBs start and stop to make me "descramble" sectors that shouldn't be touched or not unscramble sectors that need to be. I just doubt there are any discs that do that (though for all I know, I just gave them the idea)

[JohnAStebbins]

Speaking of libdvdread, I ran into a roadblock on it when trying to get it compiled, because of 'libdl' and a 'dlopen' call not being present. I couldn't go any further than that, which was why I had to settle on using VLC for the drive authentication instead of continuing to try building lsdvd. A Google search seemed to point to those being a part of libc6 that handles dynamic linking, so I would guess the Handbrake patch for large file support wouldn't matter on that point?

And yeah, the ISO I was testing earlier was from a dual layer, a full 7.47GB. I'm not sure if I have any DVDs with content under 4GB that have encryption on them. The ones I know of under that size are discs I've authored or manipulated myself, and thus would lack encryption from the start.

EDIT: Where exactly should the spotter_open call be inserted in libdvdcss' source?
libdvdread has an internal dlopen compatibility function it will use if configure detects that the build environment doesn't have dlfcn.h. It only checks for the existence of this header file and not for the actual library libdl. So my guess is that you have this header file and not the library for some reason.

[qyot27]

libdvdread has an internal dlopen compatibility function it will use if configure detects that the build environment doesn't have dlfcn.h. It only checks for the existence of this header file and not for the actual library libdl. So my guess is that you have this header file and not the library for some reason.
Checking over my /include directories, it didn't have dlfcn.h, nor the library.

However, there is a Google Code project that supplies them:
http://code.google.com/p/dlfcn-win32/

Of course, it seems there's a problem with libdvdread needing older versions of libtool and autoconf than those the environment came with.

[spotter]

wondering if anyone else has tried this? I've now made multiple rips by doing this

1) start dvd w/ VLC to authenticate disk
2) rip disk w/ scrambled sectors with imgburn (it will complain about css scrambled sectors, haven't really tried it with discs with invalid sectors, probably have to use it in a mode where it just ignores invalid sectors)
3) descramble sectors using my program.

seems to work great. I may be naive about this, but I really think the only way to defeat my program's naive approach would be to interleave real IFO data with fake VOB data such that when I try to descramble a supposed scrambled VOB block, I'm really corrupting a valid IFO block.

[qyot27]

I used ImgBurn to do some testing also, but like you said, not with any discs with bad sectors. I don't know how its Read abilities would fare against such discs, some elaboration on that would be needed (I do have a copy of Leap Year, though, so I can test that...apparently our standalone DVD players have trouble playing it too). I did, however, find out that once VLC gets the authentication step done, it doesn't need to remain open for the rescue operation to work.

If it turns out ImgBurn can't cope with bad sectors, at least there is ddrescue to fall back on. Non-Cygwin users can still grab that package from Cygwin's FTP repository and use it sans the full Cygwin environment, though (which is exactly what I did). The ddrescue package is here (ftp://ftp.gtlib.gatech.edu/pub/cygwin/release/ddrescue/), while the Cygwin system dependency package is here (ftp://ftp.gtlib.gatech.edu/pub/cygwin/release/cygwin/). Unpack both of them to C:\WINDOWS\cygwin or something, and then add C:\WINDOWS\cygwin\usr\bin to Windows' PATH variable (which can be done through Control Panel->System, regedit, or even a third-party tool (http://lingo.atspace.com/pathed.html)).

EDIT 11/11/10: Ah, seems that I was a tad mistaken about what's required for ddrescue to work. Some regular support dlls are necessary too (cyggcc, for instance). The reason I didn't realize this before is that those dlls were already in my PATH because they were sitting in C:\Program Files\Subversion. As I took that SVN installation from a Cygwin environment, I'd carried some of those dlls over already.

[spotter]

I used ImgBurn to do some testing also, but like you said, not with any discs with bad sectors. I don't know how its Read abilities would fare against such discs, some elaboration on that would be needed (I do have a copy of Leap Year, though, so I can test that...apparently our standalone DVD players have trouble playing it too). I did, however, find out that once VLC gets the authentication step done, it doesn't need to remain open for the rescue operation to work.

yea, same really with lsdvd, authentication only gets removed when disc is ejected.

If it turns out ImgBurn can't cope with bad sectors, at least there is ddrescue to fall back on. Non-Cygwin users can still grab that package from Cygwin's FTP repository and use it sans the full Cygwin environment, though (which is exactly what I did). The ddrescue package is here (ftp://ftp.gtlib.gatech.edu/pub/cygwin/release/ddrescue/), while the Cygwin system dependency package is here (ftp://ftp.gtlib.gatech.edu/pub/cygwin/release/cygwin/). Unpack both of them to C:\WINDOWS\cygwin or something, and then add C:\WINDOWS\cygwin\usr\bin to Windows' PATH variable (which can be done through Control Panel->System, regedit, or even a third-party tool (http://lingo.atspace.com/pathed.html)).

the main advantage of the imgburn approach is that its layer aware and therefore on burning of the css descrambled disc wont have to guess where the layer break should be, though from my limited experience its also able to do a very good job guessing where it should be.

also: my video card (8800GT) just went kaput over the weekend (have a 7900GS to fall back on, but not putting computer back in place till I get a replacement), so wont be doing much windows based testing. pretty comfortable with its ability in linux, though until its used heavily, wont trust that there are no bugs in it.

[spotter]

so I just tried (/in process for imgburn, but passed the bad parts) a DVD with invalid sectors with both ddrescue and imgburn. By telling imgburn to not retry invalid sectors, it had read errors on the exact same locations, and I'll do an md5sum on both images when imgburn is done, but my guess is that imgburn can be used to create the images as well. the main negative is that if you do have a scratched DVD, imgburn can't fill in the blanks from a different DVD without the scratches in those locations (very useful for recovering DVD images, which is how this project started in the first place)

[spotter]

same md5sum.

[boykillsworld]

Am I looking at this wrong because I generally just use

mplayer /dev/sr0 -dumpstream -dumpfile ${DVD_NAME}.iso

I've ripped about 120 DVD with this. I had two or three that didn't work so just used dvdfab in wine. Granted this is just what I feed handbrake with so I haven't checked the functionality of the iso. Vobcopy is terrible as I used to use it in my script. If your program really can get them all this would be a nice replacement but I don't think ddrescue works with all DVD. I often find that sometimes I am forced to just grab the movie with dvdfab as nothing in linux can grab the image.

[spotter]

why do you think ddrescue would fail? ddrescue will read all blocks of data that linux can read, if linux can't read it and it's needed to play the DVD, then it wont be playable correctly. Or let me rephrase, I'm more convinced about the ddrescue part of my methodology than my own program.

[boykillsworld]

Well i had it fail once at first I thought it was copy protection but I think it was the scratch on the DVD. Although it didn't necessarily fail I just canceled it after it took 3 hours and was making my drive make strange noises. I also disliked the approach because until now it did not remove the drm. I could pull the movie itself off but I could never make an image. It's probably a one off but if you try to rip an entire DVD the likelihood of a damaged section or being tripped up with a copy protection is substantially increased as supposed to just the movie itself. I do think the movie studios are running out of tricks but I've been trying forever to find a completely reliable way to copy just the movie on linux.

Keep up the good work as this may be the most reliable approach I've seen yet but the basis of a clean readable DVD may not be a reality.

[spotter]

Well, my approach would work with a dirty DVD as well and be no worse than any other approach, i.e. we'll just skip blocks that are scratched and can't be read and so will any other approach. The main reason I use ddrescue is that I was coming from a position where I was trying to rescue scratched DVDs so model was to rescue as much as I could and then borrow library DVD to fill in the blanks, as don't need library DVD for a lot to fill in the blanks, a pretty quick process.

[setarip_old]

@spotter

It looks like "The Last Airbender" would be an ideal DVD to test with your ripping tool...

[spotter]

I assume you mean the movie, not the TV show? might take awhile to get, 500+ holds on it in the library (though it looks like they might have ordered 300+ copies, so might cycle fast), I'll put it on hold as soon as a spot in my queue frees up, presumambly tomorrow.

[xenex]

So, since everyone seems to have problems with "The Last Airbender" with its 'latest and greatest' protection, I decided to give it a go with spotter's method. Complete success. In Linux I used ddrescue to create the ISO, which took 15 minutes for this disc, and then used spotter's program on the ISO, which took around 5 minutes. I then mounted the ISO in Windows XP using 'Virtual Clone Drive', and it played exactly like the original DVD including all menus, special features, previews, etc. etc.

I expected it would work. I have to agree with spotter, this approach is pretty close to "unbeatable." But it is because it is such a "naive" or "simple" technique that it works so well.

These "advanced" protections like ARccOS, RipGuard, etc. really can't do much to stop this method. They basically do 3 things beyond the CSS encryption:

Thing 1: Bad / Corrupt / Unreadable Sectors

"ddrescue" covers this. It simply reads all of the readable sectors from the disc, 'blanks' the unreadable ones, and creates an ISO file of the entire disc. The IFO files on the DVD instruct a compliant DVD player to skip the bad sectors, so they make no difference.

Thing 2: Messed up UDF filesystem

spotter's program addresses this by simply treating the DVD as a block device. It doesn't look at, parse, or address the filesystem in any way. It doesn't try to "fix it" or correct it. (Although I think it could, but that's for another post). Since compliant DVD players don't use or parse the filesystem either, the messed up filesystem makes no difference. The output ISO from spotter's program will still have the same messedupiness as the original disc, but for playback purposes, it makes no difference.

Thing 3: Screwy VTS's / Tiny Cells / Empty Cells / Etc.

These all affect rippers that try to parse/follow the IFO's in order to recreate the DVD in "file mode" as a VIDEO_TS folder. spotter's approach doesn't bother with that, it just simply copies the readable data from the DVD as a block device and decrypts those blocks that are CSS encrypted. So, if the DVD will play on a compliant DVD player (either Stand-Alone or in Software) then the ISO created by spotter's approach will also. QED.

This has the potential to be put together as a very nice Linux DVD Ripper package that defeats all known protections. And I have some ideas on how to do that as well. I shall post more on that later.

[spotter]

I should note that I do read the file system a little (though as ISO9660), in the sense that I look for beginning and end of VOBs as don't want to "decrypt" non VOB data. So I do think my approach is "Attackable" (interleave fake VOBs with real IFOs), but my guess is that this could mess up existing players, and why it hasn't been done.

tldr: I'm very naive in determining what sectors can be CSS encrypted and that could be a problem, would need more investigation if a problem actually came up.

[xenex]

spotter-

My previous post was not meant for you so much as it was meant for those who do not understand why your approach avoids/defeats the "advanced: - LOL - new DVD protections.

If, by posting TLDR you meant my post was "Too Long, Didn't Read." Well, sorry. sometimes it takes more than a 2 second soundbite to explain things.

I have already written C code that recognizes the DVD drive, resets the AGIDs, authenticates the drive and gathers the DVD title and other information. I also know how to use this approach to use a "ddrescue" approach to read an entire DVD and fix up the UDF filesystem.

I've already coded a lot of this for my own use. If you want to do things your own way, that's fine as well. I had hoped to work with you on this. When there is a fork in the road, take it, I guess.

I really thought you would be more open to working together to make an open-source program, but what do I know.

[spotter]

you misunderstood me. I was tldr'ing my own comment. summarising in plainer english. I'd have no problem working with you.

[spotter]

xenex: an interesting idea I had would be the distribution of map files that basically instruct the reader what sectors to skip for each DVD. (example, dealing with kick ass right now that has a boat load of invalid sectors, which makes reading very very long, though could be that my DVD is bad, but wont really know until I get another one to try and fill the holes with ddrescue)

[spotter]

actually guessing that kick ass DVD that I have is a defective, time to get another copy to verify.

[Grungefuttock]

Hi Guys
I have been following this thread from the beginning and it began to dawn on me that this is how DVDShrink started. I remember the first post from DVDShink when he introduced the early version of his program (I assume a male) and how other forum members contributed to his program. The rest is history and I still use Shrink at least a couple of times a week (via WINE as I am a Linux user and have been for the last six years).
If Spotter's program can progress along the same lines then I for one will be extremely grateful for all his and the other forum member contributions to making a Linux native ripping program that can handle all the encryptions that the DVD publishers are throwing at us.

Good luck to all

Grungefuttock

[xenex]

@spotter

So, I've been looking for bugs, and trying to find a disc where your program "fails." Haven't found one yet. But I did find an "issue" - and it's not in your code - but in libdvdcss.

It seems that libdvdcss will (in rare cases) fail to crack the key for some (usually small) VOB files. In my case, I tried the recent movie "Predators" and it DOES NOT decrypt VTS_01_1.VOB.

Now, that VOB is nothing but 1 second of black, and it doesn't matter, but it might matter if it fails on other DVDs. I have a solution, though, but am still working on it. I'm not using libdvdcss at all, but am using the CSS implementation from the old (1999) css-auth code by Derek Fawcus.

If you are able, please try the "Predators" movie and see if you have same result. The R1 disc I tested is a totally standard DVD, no bad sectors, no weird protections, but it does have a key that libdvdcss won't crack.

[spotter]

not all VOBs are encrypted. With that said, it could be how I use libdvdcss (who says I'm using it correctly).

i.e. I do a dvdcss_seek() to the start of each VOB, its possible (I'm guessing), that there will be no encrypted blocks for it to crack from the start to however far it checks and therefore the css key that it will use for later dvdcss_read() calls will be blank (or wrong). A possibility would be if that happens to do continue doing normal read() and check for css scrambled blocks and if one finds one in a vob that seek() failed on to reseek to that block's location and see if dvdcss can then crack the key.

so how I would test it,

1) if dvdcss_seek() fails, note it (code really already does it, but make it a boolean variable)
2) when the variable is set and one comes across an encrypted block (bit compare after regular read()), try to dvdcss_seek() to that location.

If it still doesn't work, then I'd say we should look at and see if vlc, totem.... have any issues with those VOBs, if they don't, its indicative that I was doing something wrong.

[spotter]

ok, I now know how to calculate layer break position on the DVD

don't quite understand this code (namely why it produces the same output for layer 0 and 1), but it provides the info to calculate the same LB proposition as imgburn

#include <sys/ioctl.h>
#include <linux/cdrom.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <strings.h>

int main()
{
dvd_struct s;

int layer = 0;
int fd;

if ( (fd = open("/dev/dvd", O_RDONLY)) == -1) {
perror("failed to open /dev/dvd");
}

while (layer < 2) {
s.type = DVD_STRUCT_PHYSICAL;
s.physical.layer_num = layer;

if ( ioctl(fd, DVD_READ_STRUCT, &s) == -1) {
perror("ioctl failed");
}

printf("layer = %d\nbook_version = %d, book_type = %d, min_rate = %d, disc_size = %d, layer_type = %d, track_path = %d, nlayers = %d, track_density = %d, linear_density = %d, bca = %d, start_sector = %u, end_sector = %u, end_sector_l0 = %u\n\n", layer, s.physical.layer[layer].book_version, s.physical.layer[layer].book_type, s.physical.layer[layer].min_rate, s.physical.layer[layer].disc_size, s.physical.layer[layer].layer_type, s.physical.layer[layer].track_path, s.physical.layer[layer].nlayers, s.physical.layer[layer].track_density, s.physical.layer[layer].linear_density, s.physical.layer[layer].bca, s.physical.layer[layer].start_sector, s.physical.layer[layer].end_sector, s.physical.layer[layer].end_sector_l0);

bzero(&s, sizeof(dvd_struct));
layer++;
}
}

layer break = end_sector_l0 - start_sector + 1

[xenex]

I don't understand why you are so concerned over the Layer Break. Any writable DVD medium cannot contain as much data (per layer) as a pressed DVD, so for "DVD burning" purposes the layer break will have to be re-calculated anyway.

Here is some output from the code I've been hacking at:

xenex@MBOXL:~/Desktop/css-auth xenex$ ./tstdvd /dev/sr1

Reset of AGIDs on drive /dev/sr1 successful

This DVD is CSS encrypted

Book Version: 1
Book Type: DVD-ROM
Disc Size: 120 mm
Num Layers: 2
Track Path: OTP

Layer 1 of [2]
Min Rate: 10.08 Mbits/sec
Layer Type: 0x1
Track Density: 0.74 um/track
Linear Density: 0.293 um/bit
BCA: Present
Start Sector 0x30000
End Sector 0xfc28cf
End Sector L0 0x20ae3f

Layer 2 of [2]
Min Rate: 10.08 Mbits/sec
Layer Type: 0x1
Track Density: 0.74 um/track
Linear Density: 0.293 um/bit
BCA: Present
Start Sector 0x30000
End Sector 0xfc28cf
End Sector L0 0x20ae3f

Request AGID [1]... AGID 3
Host sending challenge: 09 08 07 06 05 04 03 02 01 00
LU sent key1: FD 14 D4 74 2A
Drive Authentic - using variant 0
LU sent challenge: 04 6C 04 6C EB FF EB FF 1C 57
Host sending key 2: 0E 13 45 F2 EB
DVD is authenticated
Received Session Key: F4 87 62 61 5A
Received Disc Key: 53 24 5A F4 76 73 7A 22 B6 1B
Drive Authenticated



(I am FAR from finished with this. Much work to be done!)

End Sector L0 0x20ae3f tells where the data ends on Layer 1. This is still kind of useless. I'd say, let's not worry about layer breaks, or burning discs, or converting this to MS Windows. Personally, I'd rather make in work in Linux, make it work for all protections, and then go from there. Despite the cries and pleas from others! I have actually been spending my time reading specs and making sure that what is reported is correct.

Here are some specs/info that you may find useful:

http://dvd.wwwdotorg.org/specs/specs.html

http://www.tinyted.net/eddie/css_auth.html

http://read.pudn.com/downloads43/ebook/147287/ATAPI-DVD_SFF8090r99.pdf

[spotter]

I'm mostly trying to figure out how to duplicate the dvd decrypter (which does generate the mds file).

some of these things aren't that important, but I'm also documenting them for myself so I can find them in the future.

[setarip_old]

Have I missed something about your project up until now? - Is playback limited to .ISO image files played on your PC - or can you also burn a disc that's playable on a standalone player?

[xenex]

@setarip_old

Isn't this just a repeat of the same question you asked, which was asked and answered on page four of this thread, here?:

http://forum.doom9.org/showthread.php?p=1454017#post1454017

This whole approach is fairly simple, really. No rocket science is needed. The output of spotter's program is an essentially an EXACT IMAGE of the original DVD with two exceptions.

1) Where the ORIGINAL DVD had bad/corrupt/unreadable sectors, ddrescue replaces them with blank/empty sectors in the ISO.

2) Where the ORIGINAL DVD had CSS-ENCRYPED sectors (VOB files) spotter's program overwrites them with CSS-DECRYPTED sectors.

That's it, basically. Two steps. Nothing else going on.

So, if the output ISO is small enough to be burned to a DVD5 SL writable medium, no problem, it will work.

If the output ISO is small enough to be burned to a DVD9 DL writable medium, it can be done, but it will absolutely require the layer break to be recalculated, probably using ImgBurn, which does a good job of it.

If the output ISO is HUGE and too large to be burned to a DVD9 DL writable medium, it can't be done at all without some 'editing.'

Personally, I NEVER burn movies to shiny discs. Do people still do this, LOL. I just save the ISO images on a drive and play them from there.

To each their own, I suppose.

[setarip_old]

@xenex

It was not answered with certainty, only speculated upon by the O.P. - and, based on your remarks in your most recent post #134 that: So, if the output ISO is small enough to be burned to a DVD5 SL writable medium, no problem, it will work.

If the output ISO is small enough to be burned to a DVD9 DL writable medium, it can be done, but it will absolutely require the layer break to be recalculated, probably using ImgBurn, which does a good job of it.You too, are speculating. Since "spotter" has stated that he is reluctant to provide a Windows version for the masses (including me) who could readily test the theory, perhaps he, or another Linux user, could simply burn a couple of such DVD backups to disc (e.g. one single layer sample and one double layer sample) and report the results of actual testing...

BTW, your remark regarding your personal preferences:Personally, I NEVER burn movies to shiny discs. Do people still do this, LOL.Has nothing to do with the question at hand, namely, will such a saved backup play properly from a burned disc on a standalone player?

[spotter]

I don't have a standalone player. all I have is my linux laptop where I do most of my real work, and my toy windows desktop (don't even own a TV!). I guess if I was thinking in advance, I could have burned a DVD5 test (have a handful of DVDs that would fit) and tried it out on my parents standalone players when I was there for thanksgiving weekend, but didn't think to.

with that said, I agree with xenex, I would be shocked if my decrypted ISO didn't work on a standalone player. It's just a DVD without any CSS.

The one thing that would amuse me, I'm wondering if there are any DVD players that basically implement libdvdcss, i.e. instead of doing the "proper" way to decode the DVDs, they basically ignore the "unwritable" area of the DVD and just automatically crack each DVD, and then even if you would burn a DVD with css scrambled sectors, it would still play perfectly.

[spotter]

also, if you really wanted to test this.

http://www.vmware.com/products/player/
http://www.vmware.com/appliances/directory/va/752043

[setarip_old]

@spotter also, if you really wanted to test this.As I mentioned previously, I know nothing about Linux, nor do I understand how I would use "VMWare" to test your program that I've never seen, nor have the ability to build.

I would be shocked if my decrypted ISO didn't work on a standalone player. It's just a DVD without any CSS.You may discover that standalone DVD players interpret damaged sectors differently than they do zero-filled sectors...

[spotter]

ok, and I'm not really a windows progammer :)

[qyot27]

@spotter As I mentioned previously, I know nothing about Linux, nor do I understand how I would use "VMWare" to test your program that I've never seen, nor have the ability to build.
VMWare is a virtual machine - you use it to install a copy of Linux to a virtual hard drive file without touching the existing Windows setup. And then you bring up the virtual machine (VMWare) and use a different OS (Linux) without logging out of the one you're currently in (Windows). From what I understand, Windows 7 uses a virtual machine approach if you use XP mode.

Personally, I'd use VirtualBox instead of VMWare, but it's the same premise. Well, actually, I'd forgo using a virtual machine at all and just install Ubuntu inside Windows using their Wubi installer if I didn't want to properly set the machine up to dual-boot. If you do it through Wubi there's little performance penalty (unlike a virtual machine, which is highly dependent on your hardware setup if you want to get decent performance out of it), and it can be uninstalled from Windows just as easily as it was installed.

If you're interested, I can provide step-by-step instructions through the whole process, either with Linux or with Windows.

The install for Ubuntu with Wubi (and the rest of the setup process the first time you go into Ubuntu) is pretty self-explanatory, so I don't think I'd need to go too in-depth on that, but to show how to build the program using a basic install I can do pretty easily (and it just requires some copying and pasting, not anything big; no real Linux knowledge needed for it either).

I could also explain how to do it with MinGW and MSys so that you can build a native Windows version without ever installing Ubuntu, but that's a longer process because it's more involved to get everything set up correctly. On Ubuntu you can do it with a single copy-pasted command.

[setarip_old]

@qyot27

Hi!

Thank you for your kind and generous offer - but it would be far simpler if someone already versed in Linux would use the Linux tool created by "spotter" to burn a backup disc (or two) and see if it plays on a standalone DVD player...

[xenex]

@setarip_old

I'd actually be happy to test the "burn a disc" issue, but the problem on my end is that I don't have any DVDs whose resultant ISO would be small enough to fit on a SL disc, and I don't own any writeable DL discs. I can't just go and buy some either, easily, as the small town in which I live has only a Wal-Mart which only sells crappy Memorex media, and does not even offer DL discs!

I'll ask Santa for some and see if they appear in my stocking. :)

But really, I think the whole "burn a disc" thing is getting the cart before the horse, honestly. Spotter's 'approach' is solid, but there are some problems in the implementation. Using the "ddrescue" program on DVDs that have many bad/corrupt sectors can take a LONG LONG TIME.

Just earlier, I got the new "Twilight: Eclipse" DVD and it has a RIDICULOUS UNHOLY number of intentional bad sectors. It took ddrescue hours to read it.

And, spotter's program isn't performing the CSS decryption in the optimal way. It's not a bug, per se, but an implementation issue.

@spotter

Here's what I mean, from libdvdcss.c:

* \li \b DVDCSS_METHOD: sets the authentication and decryption method
* that \e libdvdcss will use to read scrambled discs. Can be one
* of \c title, \c key or \c disc.
* - \c key is the default method. \e libdvdcss will use a set of
* calculated player keys to try and get the disc key. This can fail
* if the drive does not recognize any of the player keys.
* - \c disc is a fallback method when \c key has failed. Instead of
* using player keys, \e libdvdcss will crack the disc key using
* a brute force algorithm. This process is CPU intensive and requires
* 64 MB of memory to store temporary data.
* - \c title is the fallback when all other methods have failed. It does
* not rely on a key exchange with the DVD drive, but rather uses a
* crypto attack to guess the title key. On rare cases this may fail
* because there is not enough encrypted data on the disc to perform
* a statistical attack, but in the other hand it is the only way to
* decrypt a DVD stored on a hard disc, or a DVD with the wrong region
* on an RPC2 drive.

Since spotter is trying to decrypt an ISO on the hard disc, the two (better) methods 'key' and 'disc' are unavailable. So his program uses the final resort 'title' method which fails badly on small/short VOB files.

I've tested the heck out of this and know what is happening. And I'm working on a solution, but I only have an hour or two per day to work on it. I really am working on it, though, even if it will start out as a "proof of concept."

So, I hope that's clear as to why "burn a disc" is on my back burner. I would rather get this whole program sorted out in Linux, and working well, and solid first. And I really am working on it. I only have a few hours a day to spend, but I really am trying. I want a good Linux DVD ripper, I've always wondered why no one else has done it before, but spotter's idea inspired me.

[setarip_old]

So, I hope that's clear as to why "burn a disc" is on my back burner.Actually, it would be more interesting to hear from someone who HAS been able to actually test the performance of "spotter's" program with a burned disc (with both a standalone player and a software player - and, if it should come to pass, if someone releases a Windows version, so that MANY users can simply burn a disc or two to test it...)

[spotter]

@setarip_old

I'd actually be happy to test the "burn a disc" issue, but the problem on my end is that I don't have any DVDs whose resultant ISO would be small enough to fit on a SL disc, and I don't own any writeable DL discs. I can't just go and buy some either, easily, as the small town in which I live has only a Wal-Mart which only sells crappy Memorex media, and does not even offer DL discs!

I'll ask Santa for some and see if they appear in my stocking. :)

But really, I think the whole "burn a disc" thing is getting the cart before the horse, honestly. Spotter's 'approach' is solid, but there are some problems in the implementation. Using the "ddrescue" program on DVDs that have many bad/corrupt sectors can take a LONG LONG TIME.

Just earlier, I got the new "Twilight: Eclipse" DVD and it has a RIDICULOUS UNHOLY number of intentional bad sectors. It took ddrescue hours to read it.

And, spotter's program isn't performing the CSS decryption in the optimal way. It's not a bug, per se, but an implementation issue.

@spotter

Here's what I mean, from libdvdcss.c:

* \li \b DVDCSS_METHOD: sets the authentication and decryption method
* that \e libdvdcss will use to read scrambled discs. Can be one
* of \c title, \c key or \c disc.
* - \c key is the default method. \e libdvdcss will use a set of
* calculated player keys to try and get the disc key. This can fail
* if the drive does not recognize any of the player keys.
* - \c disc is a fallback method when \c key has failed. Instead of
* using player keys, \e libdvdcss will crack the disc key using
* a brute force algorithm. This process is CPU intensive and requires
* 64 MB of memory to store temporary data.
* - \c title is the fallback when all other methods have failed. It does
* not rely on a key exchange with the DVD drive, but rather uses a
* crypto attack to guess the title key. On rare cases this may fail
* because there is not enough encrypted data on the disc to perform
* a statistical attack, but in the other hand it is the only way to
* decrypt a DVD stored on a hard disc, or a DVD with the wrong region
* on an RPC2 drive.

Since spotter is trying to decrypt an ISO on the hard disc, the two (better) methods 'key' and 'disc' are unavailable. So his program uses the final resort 'title' method which fails badly on small/short VOB files.

I've tested the heck out of this and know what is happening. And I'm working on a solution, but I only have an hour or two per day to work on it. I really am working on it, though, even if it will start out as a "proof of concept."

So, I hope that's clear as to why "burn a disc" is on my back burner. I would rather get this whole program sorted out in Linux, and working well, and solid first. And I really am working on it. I only have a few hours a day to spend, but I really am trying. I want a good Linux DVD ripper, I've always wondered why no one else has done it before, but spotter's idea inspired me.

ah, make sense, I had to resort to ddrescue due to the original goal was to recover scratched media, but a decrypter/ddrescue combo tool would have access to all the keys. But it makes sense while the final resort method wouldn't work because there's not enough data to crack the key.

[mc2man]

burn a backup disc (or two) and see if it plays on a standalone DVD player...
It works quite fine, no issues to speak of other than a small 'oddity' with one title as far as LB
There are certainly some limitations to the method though there shouldn't be any problem with any disk that can playback in linux. (have seen a very small number that won't.

Due to having to 'read' thru bad sectors rather than exclude i won't consider any title that has more than 5 - 7 thousand unreadable sectors, like most sony protections. A typical ARccOS disk may take several hours - hardly worth it.
Here with a drive that recovers nicely from read errors it's about 1.2 sec.'s/block of 16 sectors so with most of the current protections time isn't an issue.
The structure protection, just like on encrypted dumps to .iso's on a hdd, is irrelevant.

Used 2 titles, both with less than 1000 bad sectors and some form of sp. - The Last Airbender which also misrepresents size and Ratatouille because it needed backing up and has a fair # of extras, easter eggs, ect.

For dumping used dvdisaster instead, used cdemu to mount the .iso's both before and after using spotter's code, the css encryption was nicely removed.

As far as burning to dl, again no real issue. While the rat movie could have been extracted to file for imgburn to create an iso and mds, didn't bother. It gave 1 choice for Lb and the burned disk works fine in a standalone in all regards.
(did extract later to ck, in that case imgburn showed 4 possible

The airbender iso had to be burned as is, in this case imgburn showed 9 possible Lb's, all at the same LBA but several different VTS's, PGC's, chapters, ect.
One seemed clearly obvious, not sure if it would have mattered.
Again the disk plays perfectly in a standalone.

[spotter]

yes, I'm dealing right now with Kick Ass, it seems to have a huge (100s of MBs of invalid sectors). Pondering if anything can be done to make it better.

Another thing I'm wondering, for non whole disk decryption, why hasn't anyone combined DVD Player w/ Menu UI to initiate the decryption i.e. use libdvdnav to initiate the dvd vm and just read blocks one after the other as the dvdnav says to. Only thing I can see going wrong with this approach is that perhaps you don't read every "track", but I'd think that would occur afterwards in the demuxer, but could be wrong.

[spotter]

so did some timing measurements on my laptop for the invalid sectors in kick ass.

I read each 2048 byte block in turn and measured how long it took (mostly for measuring how long it takes invalid sectors to return). doesn't seem to make a difference with or without O_DIRECT. Obviously way too long for a disk with a huge numer of invalid sectors.

[xenex]

@spotter

So, I modified your program (a bit) and I like this output better - this is the same "Kick Ass" DVD you are working on. If you can verify that the sectors match, it will help me:


xenex@MBOXL:~/Desktop$ ./a.out /dev/sr0 -test

total_blocks = 4169920

video_ts.ifo : 0000416 -> 0000433 : (18 Blocks)
video_ts.vob : 0000464 -> 0000520 : (57 Blocks)
video_ts.bup : 0000544 -> 0000561 : (18 Blocks)

vts_01_0.ifo : 0158085 -> 0158205 : (121 Blocks)
vts_01_0.vob : 0158206 -> 0313954 : (155749 Blocks)
vts_01_1.vob : 0313955 -> 0838241 : (524287 Blocks)
vts_01_2.vob : 0838242 -> 1362528 : (524287 Blocks)
vts_01_3.vob : 1362529 -> 1886815 : (524287 Blocks)
vts_01_4.vob : 1886816 -> 2411102 : (524287 Blocks)
vts_01_5.vob : 2411103 -> 2930154 : (519052 Blocks)
vts_01_0.bup : 2930155 -> 2930275 : (121 Blocks)

vts_02_0.ifo : 2930276 -> 2930290 : (15 Blocks)
vts_02_0.vob : 2930291 -> 2930347 : (57 Blocks)
vts_02_1.vob : 2930348 -> 3338552 : (408205 Blocks)
vts_02_0.bup : 3338553 -> 3338567 : (15 Blocks)

vts_03_0.ifo : 3338568 -> 3338576 : (9 Blocks)
vts_03_0.vob : 3338577 -> 3338633 : (57 Blocks)
vts_03_1.vob : 3338634 -> 3387378 : (48745 Blocks)
vts_03_0.bup : 3387379 -> 3387387 : (9 Blocks)

vts_04_0.ifo : 3387388 -> 3387396 : (9 Blocks)
vts_04_0.vob : 3387397 -> 3387453 : (57 Blocks)
vts_04_1.vob : 3387454 -> 3412613 : (25160 Blocks)
vts_04_0.bup : 3412614 -> 3412622 : (9 Blocks)

vts_05_0.ifo : 3412623 -> 3412631 : (9 Blocks)
vts_05_0.vob : 3412632 -> 3412688 : (57 Blocks)
vts_05_1.vob : 3412689 -> 3463087 : (50399 Blocks)
vts_05_0.bup : 3463088 -> 3463096 : (9 Blocks)

vts_06_0.ifo : 3463097 -> 3463105 : (9 Blocks)
vts_06_0.vob : 3463106 -> 3463162 : (57 Blocks)
vts_06_1.vob : 3463163 -> 3513561 : (50399 Blocks)
vts_06_0.bup : 3513562 -> 3513570 : (9 Blocks)

vts_07_0.ifo : 3513571 -> 3513579 : (9 Blocks)
vts_07_0.vob : 3513580 -> 3513636 : (57 Blocks)
vts_07_1.vob : 3513637 -> 3556439 : (42803 Blocks)
vts_07_0.bup : 3556440 -> 3556448 : (9 Blocks)

vts_08_0.ifo : 3556449 -> 3556457 : (9 Blocks)
vts_08_0.vob : 3556458 -> 3556514 : (57 Blocks)
vts_08_1.vob : 3556515 -> 3577791 : (21277 Blocks)
vts_08_0.bup : 3577792 -> 3577800 : (9 Blocks)

vts_09_0.ifo : 3577801 -> 3577809 : (9 Blocks)
vts_09_0.vob : 3577810 -> 3577866 : (57 Blocks)
vts_09_1.vob : 3577867 -> 3598148 : (20282 Blocks)
vts_09_0.bup : 3598149 -> 3598157 : (9 Blocks)

vts_10_0.ifo : 3598158 -> 3598166 : (9 Blocks)
vts_10_0.vob : 3598167 -> 3598223 : (57 Blocks)
vts_10_1.vob : 3598224 -> 3620886 : (22663 Blocks)
vts_10_0.bup : 3620887 -> 3620895 : (9 Blocks)

vts_11_0.ifo : 3620896 -> 3620904 : (9 Blocks)
vts_11_0.vob : 3620905 -> 3620961 : (57 Blocks)
vts_11_1.vob : 3620962 -> 3622771 : (1810 Blocks)
vts_11_0.bup : 3622772 -> 3622780 : (9 Blocks)

vts_12_0.ifo : 3622781 -> 3622789 : (9 Blocks)
vts_12_0.vob : 3622790 -> 3622846 : (57 Blocks)
vts_12_1.vob : 3622847 -> 3630323 : (7477 Blocks)
vts_12_0.bup : 3630324 -> 3630332 : (9 Blocks)

vts_13_0.ifo : 3630333 -> 3630341 : (9 Blocks)
vts_13_0.vob : 3630342 -> 3630398 : (57 Blocks)
vts_13_1.vob : 3630399 -> 3631793 : (1395 Blocks)
vts_13_0.bup : 3631794 -> 3631802 : (9 Blocks)

vts_14_0.ifo : 3631803 -> 3631811 : (9 Blocks)
vts_14_0.vob : 3631812 -> 3631868 : (57 Blocks)
vts_14_1.vob : 3631869 -> 3639345 : (7477 Blocks)
vts_14_0.bup : 3639346 -> 3639354 : (9 Blocks)

vts_15_0.ifo : 3639355 -> 3639361 : (7 Blocks)
vts_15_1.vob : 3639363 -> 3639432 : (70 Blocks)
vts_15_0.bup : 3639433 -> 3639439 : (7 Blocks)

vts_16_0.ifo : 3639440 -> 3639452 : (13 Blocks)
vts_16_1.vob : 3639454 -> 3657793 : (18340 Blocks)
vts_16_0.bup : 3657794 -> 3657806 : (13 Blocks)

vts_17_0.ifo : 3657807 -> 3657824 : (18 Blocks)
vts_17_1.vob : 3657826 -> 3694225 : (36400 Blocks)
vts_17_0.bup : 3694226 -> 3694243 : (18 Blocks)

vts_18_0.ifo : 3694244 -> 3694530 : (287 Blocks)
vts_18_1.vob : 3694532 -> 3891021 : (196490 Blocks)
vts_18_0.bup : 3891022 -> 3891308 : (287 Blocks)

vts_19_0.ifo : 3891309 -> 3891344 : (36 Blocks)
vts_19_1.vob : 3891346 -> 3989415 : (98070 Blocks)
vts_19_0.bup : 3989416 -> 3989451 : (36 Blocks)

vts_20_0.ifo : 3989452 -> 3989494 : (43 Blocks)
vts_20_0.vob : 3989495 -> 4027588 : (38094 Blocks)
vts_20_1.vob : 4027589 -> 4138748 : (111160 Blocks)
vts_20_0.bup : 4138749 -> 4138791 : (43 Blocks)

[spotter]

basically the same (without modifying my program to output every file, a cursory look at the VOBs seem to indicate same size

total_blocks = 4169920
video_ts.vob: 464->520 (57 blocks)
vts_01_0.vob: 158206->313954 (155749 blocks)
vts_01_1.vob: 313955->838241 (524287 blocks)
vts_01_2.vob: 838242->1362528 (524287 blocks)
vts_01_3.vob: 1362529->1886815 (524287 blocks)
vts_01_4.vob: 1886816->2411102 (524287 blocks)
vts_01_5.vob: 2411103->2930154 (519052 blocks)
vts_02_0.vob: 2930291->2930347 (57 blocks)
vts_02_1.vob: 2930348->3338552 (408205 blocks)
vts_03_0.vob: 3338577->3338633 (57 blocks)
vts_03_1.vob: 3338634->3387378 (48745 blocks)
vts_04_0.vob: 3387397->3387453 (57 blocks)
vts_04_1.vob: 3387454->3412613 (25160 blocks)
vts_05_0.vob: 3412632->3412688 (57 blocks)
vts_05_1.vob: 3412689->3463087 (50399 blocks)
vts_06_0.vob: 3463106->3463162 (57 blocks)
vts_06_1.vob: 3463163->3513561 (50399 blocks)
vts_07_0.vob: 3513580->3513636 (57 blocks)
vts_07_1.vob: 3513637->3556439 (42803 blocks)
vts_08_0.vob: 3556458->3556514 (57 blocks)
vts_08_1.vob: 3556515->3577791 (21277 blocks)
vts_09_0.vob: 3577810->3577866 (57 blocks)
vts_09_1.vob: 3577867->3598148 (20282 blocks)
vts_10_0.vob: 3598167->3598223 (57 blocks)
vts_10_1.vob: 3598224->3620886 (22663 blocks)
vts_11_0.vob: 3620905->3620961 (57 blocks)
vts_11_1.vob: 3620962->3622771 (1810 blocks)
vts_12_0.vob: 3622790->3622846 (57 blocks)
vts_12_1.vob: 3622847->3630323 (7477 blocks)
vts_13_0.vob: 3630342->3630398 (57 blocks)
vts_13_1.vob: 3630399->3631793 (1395 blocks)
vts_14_0.vob: 3631812->3631868 (57 blocks)
vts_14_1.vob: 3631869->3639345 (7477 blocks)
vts_15_1.vob: 3639363->3639432 (70 blocks)
vts_16_1.vob: 3639454->3657793 (18340 blocks)
vts_17_1.vob: 3657826->3694225 (36400 blocks)
vts_18_1.vob: 3694532->3891021 (196490 blocks)
vts_19_1.vob: 3891346->3989415 (98070 blocks)
vts_20_0.vob: 3989495->4027588 (38094 blocks)
vts_20_1.vob: 4027589->4138748 (111160 blocks)

though i'm not finishing scanning for bad blocks, and once the initial scan is done, will go over it once more. (currently have about 243MB of "bad" blocks with 71 areas that ddrescue still has to thoroughly go through)

[spotter]

so xenex, I've now come across a bunch of DVDs that have the non decrypting issue you encountered, and you're right that it probably work if one had the disc's key, but I'm still wondering if there's no way to decrypt those small ones without the disc's key. or perhaps to read the disc key during copying and then to somehow set it in my program.