Finally handling BD+ (?) [Archive] - Page 11

bugnotme

6th December 2008, 02:34

Now that's some good news! :)

Happy debugging and please make sure you don't implement anything undocumented just to make it work for this snapshot package (like Schluppo did it sometimes). If you have any questions make a list and send it to me. ;)

In the meantime i've studied some attacks on the RSA scheme with small exponents. Because of the content code running on a virtual machine and therefor being very slow the license administration has chosen a very small public key exponent e = 3 in order to improve the speed. It is well know that this leaks half of the most significant bits of the private key d.

The result is:
d'=
5CB9BF8C681B23E2E923E0EAFD02129E20D0137B1F4E411829A2D76F3E004EE9
EFC2FF78D3870835A4E1CB06EEF61911CA0D5F049F2135937834A5566E7520E0
2701D405BE8A18EBB036BBBE19DDA674662EF16CBBBBC72F2C49A93C4417586F
A1E1220331335DA182CDE42B41BD443527EBEFBB7C206268F526191EB2CE05B9
63506A7C9BD0CBFD568DE0AF77479731B050965B87E574CAAFAAF3A3DEBA1CE5
where d' is the approximation of d satisfying:
| d' - d | <= 2 * square_root( N )
Where N is the public key modulus of the BD+ license administration. Finding the private key of the license administration would be a major step towards a break of BD+.

What did you do to calculate d'?

yippiekayee

6th December 2008, 03:13

I don't know which version of Jumper Oopho2ei tested against - so I took my US copy of Jumper for a quick spin. The 0.1.3 debugger creates a convtable with no problems.. however the 0.1.4 debugger throws an ArrayIndexOutOfBoundsException. The full error is:
java.lang.ArrayIndexOutOfBoundsException: -1621801643
at bdvm.vm.conv_tab_main.setConversionTable(conv_tab_main.java:43)
at bdvm.vm.decoder.getConversionTable(decoder.java:40)
at bdvm.vm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.ja
va:658)
at bdvm.vm.BDVM.step(BDVM.java:338)
at bdvm.debugger.Debugger.step(Debugger.java:740)
at bdvm.debugger.Debugger.run(Debugger.java:867)
at bdvm.debugger.Debugger.actionPerformed(Debugger.java:342)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
.... more internal java stuff

The commandline output looks like this:
$ Volume ID set to: 50 4D B5 A0 FC 6F 18 05 47 DF 27 A0 A7 A9 0B E7
Loading E:\/BDSVM/00000.svm ...
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 6)
[I] TRAP_DebugLog: 2008-11-07 02:56:55.37 playback starts
[I] TRAP_MediaSHAFileHash: Hashing BDSVM/00002.svm
[I] TRAP_MediaSHAFileHash: Hashing AACS/MKB_RO.inf
[I] TRAP_MediaSHAFileHash: Hashing BDMV/STREAM/00002.m2ts
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 7)
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[W] TRAP_DeviceAccess not implemented!
[Event #00000002] 0110 ( 00000000, 00000001 )
[I] TRAP_LoadContentCode: Loading BDSVM/00002.svm (block 0)
[I] TRAP_GetConversionTable: Got subtables 2,4,19,20,21,33,34,35,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,
... many events
....
[I] TRAP_LoadContentCode: Loading BDSVM/00002.svm (block 1)
[I] TRAP_GetConversionTable: Got subtables

And some good news.. the entire Die Hard series (US version) decrypted fine and is byte identical with AnyDVD HD.

yippiekayee

6th December 2008, 04:14

it seems convtabd.exe actually works - however, it doesn't seem to like Prison Break Season 3 Disk 4 (and sorry for reporting the problem with that disc.. I indeed picked the wrong convtable)..
I ran convtabd.exe as follows
convtabd.exe -d e: -I 14870BDEF23FD4B9F2E53150B1414041 -s convtab-pbs3d4.bin
And I get an infinite number of
[dlx] unknown trap 00008020:32800.
messages.

Oh, and I'm running this thing natively... cygwin was no good.. didn't offer me access to my BD drive, refused to allow me to create any directory under /cygwin and when selecting another directory mounting would fail.

Accident

6th December 2008, 04:19

That is interesting. It should be just replying with NOT_SUPPORTED for that trap, but I suspect that it has already gone wrong if it ends up calling it.

I may need the files required for it to work out what the problem is.

yippiekayee

6th December 2008, 12:50

Alright here you go.

yippiekayee

6th December 2008, 14:00

Some good news and some bad news:
Jumper (US) and Independence Day (US) both decrypt fine and are byte identical with AnyDVD HD. And I redid Prison Break S3 Disc 4 and it is now okay as well.
However, the convtable generated by libblu-ray differs for ID4. And for Jumper, libbluray takes several minutes and then returns nothing at all.. there's also no output whatsoever on the commandline even with -v 64.

loo3aem3ON

6th December 2008, 15:00

I don't know which version of Jumper [...] - so I took my US copy of Jumper for a quick spin. The 0.1.3 debugger creates a convtable with no problems.. however the 0.1.4 debugger throws an ArrayIndexOutOfBoundsException.
That's because the subtable already existed with removed bogus descriptors. I use the length of the existing (newly created) subtable object to navigate to the next subtable. This will obviously fail if the new subtable is longer than the existing subtable object because it contains the bogus descriptor. Because the events and the conversion table are still not fully understood it is difficult to find a good implementation. Until recently we didn't even know these bogus descriptors existed.
Anyway i decided to postpone segment decryption/filtering until all parts of the conversion table have been received which should solve this problem. Here is the new official release: BDVM debugger 0.1.4 (http://uploaded.to/?id=7awa5o)

yippiekayee

6th December 2008, 15:31

The convtable also differs for Home of the Brave (US) and Ice Age (US) - both are properly being decrypted with the convtable from the debugger. The convtable also differs for Sunshine (DE).. but I haven't checked if the debugger convtable is correct.

dirio49

6th December 2008, 18:40

Is there any difference between contabd and contab? because in linux I only get contab.

yippiekayee

6th December 2008, 21:20

I also did the Simpsons Movie (DE).. same story with libbluray.. it creates a convtable but it's not identical with the one from the debugger.
And with Dr. No (US) the program just sits there for minutes maxing out one ofmy cores but otherwise doing nothing (no output, no convtable).
Since I haven't had one single success with lib-bluray I'm going to stop creating tables now.. I think there's something seriously wrong with the software at this point.. maybe it works on Linux but it definitely doesn't work properly on Windows.

loo3aem3ON

6th December 2008, 23:26

I also did the Simpsons Movie (DE).. same story with libbluray.. it creates a convtable but it's not identical with the one from the debugger.
Can you tell us what bytes differ? For example appending a few garbage bytes to a correct conversion table (fix-up table) doesn't make it invalid. If there is a problem with libblueray then we need to solve it.

Accident: Adding support to create diffarchive snapshots for the debugger is on my "ToDo" list. This will probably help you if you can't figure out the problems quickly. :)

yippiekayee

7th December 2008, 00:30

I don't really know how I could tell the difference other than ripping and md5sum'ing the results and that's just too time consuming - going through all movies once takes maybe 2.5 hours per disc and another rip would bring that up to 3.5+ hours. The best I can do is post the 6 convtables I got from libbluray and the corresponding debugger convtables (which are all verified against AnyDVD HD).

Accident

7th December 2008, 01:20

I can't guarantee the binary conv_tab.bin files would be identical. I will spend some more time with the Windows version, I need to implement the hash.db file first, since I can not fit 40GB files to try, (Windows does not do sparse files).

Other considerations is that it should work on big-endian, but has not been tested. It should work on 64bit, but has not been tested.

yippiekayee: I appreciate the hard work you have done, and I am committed to making it work for everything.

yippiekayee

7th December 2008, 02:00

Turns out Bender's Game is actually not using MKBv9 so that made the whole process of trying it out a little easier. Anyway, the convtable was created without error though the output of the debugger is quite different from any other movie.. and it is half the size of most other tables. ConvTableView has a lot of messages though like patch adresses being smaller than the previous one and patch exceeding TS packet boundaries. Decryption also went fine but about 2 minutes in (TMT wouldn't play the thing in movie mode, PowerDVD started using a lot of CPU power immediately and played choppy, and went to slideshow mode when I moved around the timeline) the picture started getting corrupted so I conclude that the movie wasn't properly decrypted.

My next stop will be Firefly but that is definitely MKBv9 so I'll have to go for a two step process.

@Accident: I'm on 32bit Windows (XP).

MickJT

7th December 2008, 04:39

My next stop will be Firefly but that is definitely MKBv9 so I'll have to go for a two step process.

Is this process removing AACS (only) with AnyDVD HD, and then BD+ with libbluray?

That won't necessarily work, as some of the newer BD+ require intact "with AACS" .m2ts files, and AACS and BD+ need to be removed at the same time.

KenD00

7th December 2008, 12:27

Thats not a problem right now because you can run the BDVM Debugger as stand alone application. The workflow is to use it on the still encrypted disc, and then decrypt the disc with DumpHD using the produced Conversion Table (or any other Ripper and applying the BD+ removal as second step).

:rolleyes:

Rupan

7th December 2008, 13:03

@Accident

Would it be helpful for me to test libbluray on 64-bit Linux?

EDIT: I've uploaded conversion tables produced on 64-bit Linux with libbluray and the bdvm java debugger. Both tables are exactly the same byte size but their contents differ greatly. I'm waiting on md5sum to finish, which may not be really done for a day or so, and will get back to you when it is complete.

These conversion tables were produced using Live Free or Die Hard (US).

http://www.sendspace.com/file/2s64n3

yippiekayee

7th December 2008, 17:35

yippiekayee

7th December 2008, 21:39

Thanks to the help of Kend00 I now managed to apply the convtable to Firefly.. and the resulting video stream is still corrupt.
There are two more discs in the set but I'm hoping that what I've posted is enough to improve the debugger to the point where it can handle the latest batch of Fox discs (I don't have any other titles and the list of titles AnyDVD HD cannot handle doesn't really include other titles that are interesting to me).

Rupan

7th December 2008, 23:21

Hmm.. so you actually decrypted the movie twice? md5suming the convtable should only take a second.. md5sum'ing a movie takes maybe 15 minutes or so if you have a decent harddisk.

I haven't managed to get DumpHD to fix BD+ on an AnyDVD HD decrypted disc - however seeing as the convtable is just as small as with Bender's Game and seeing the same events in the debugger logs I figured I'd post (http://rapidshare.com/files/171165285/Firefly-Disc1.7z.html) the BDSVM, hashes, contable and debugger output here just in case. Once I know how to use DumpHD to remove the BD+ I'll check playback and report back.

This is (again) a dump of the US version of Live Free or Die Hard. Any heavy file I/O operations on my workstation take 20-50% longer than on most other workstations. My disks are all backed with 256-bit AES encryption which consumes roughly 65% of one core during heavy I/O.

Here is a list of files that differ between the AnyDVD dump and the dump created by DumpHD+bdvmdbg. All of the video streams (m2ts) are byte identical:

./CERTIFICATE/app.discroot.crt
./BDMV/MovieObject.bdmv
./BDMV/PLAYLIST/* (about 90% of them anyway)
./BDMV/JAR/00000.jar
./BDMV/JAR/00002.jar

which I believe are related to the region locks and removal of annoying trailers. I assume the certificate differs because it contains cryptographic signatures of all the unencrypted files on the disc and must be fixed when their content is changed.

I'll re-rip with the convtable produced by libbluray and get back with the results later today.

EDIT2: I'm attaching a copy of the 64-bit convtab linux executable built from svn revision 158. My copy of Die Hard 4 is currently being processed by DumpHD 0.5.1 with the libbluray-produced convtable. When it is finished copying I will compare the checksums again and update this post.

Rupan

8th December 2008, 01:34

I just finished dumping and checksumming Die Hard 4 with the convtable produced by libbluray. All file checksums match the dump produced by bdvmdbg's convtable except the main title (33GB: 00000.mt2s). I suspect that this may be a bug in the libbluray implementation rather than a 64-bit issue since all the other file checksums match.

loo3aem3ON

8th December 2008, 01:44

Thank you for your support. Like i previously wrote i don't like to support more BD+ protected (damaged) movies than AnyDVD-HD and more importantly i don't want this project to make such ridiculous headlines again. Several sites were reporting of a break of BD+ which never occurred and now the vendor of my player seems to have stopped releasing updates. Being 1-2 months behind SlySoft is acceptable for me.
If you want to help this project then learn how BD+ works by reading the documentations and looking at the source code. I've also started this (https://forum.doom9.org/showthread.php?t=143247) thread where you can propose your ideas about how to exploit various weaknesses of BD+ which you might have found. Your ideas could lead to an attack which really breaks BD+ but i believe this is unlikely. Finally i also saw wikipedia reporting or "BD+ circumvention" which would mean repairing the damaged m2ts files without the execution of the content code. I don't know if this is possible in general but i am quite sure AnyDVD-HD executes the content code and so do we.

KenD00

8th December 2008, 03:51

All file checksums match the dump produced by bdvmdbg's convtable except the main title (33GB: 00000.mt2s)

Which is the only file that is BD+ protected...

:rolleyes:

Accident

8th December 2008, 05:13

Phew that is a lot of bad news coming my way. Can we sum up what is broken and needs fixing. Maybe also mention what works as encouragement... :)

I assume there are titles that:

* Do not work in Debugger, nor libbluray
* Work in Debugger but not libbluray
* Work in Unix but not Windows
* Work in 32-bit but not 64-bit.

Ice Age 1 MD5
Unix : e09cc8bf331b8283462c3b8ceebe889e
Win32 : e09cc8bf331b8283462c3b8ceebe889e
Jumper MD5
Unix : 9ce4443630cd7e5fce61ddeb84400fb8
Win32 : 9ce4443630cd7e5fce61ddeb84400fb8

(I take back my comment about sparse files under NTFS, works just fine).

I can not try PBS3D4, as it hashes file in AACS/

[trap] reading 'AACS/MKB_RO.inf' at pos 0000000000000000
[trap] reading 'STREAM/00000.m2ts' at pos 00000000e9c82600
[trap] reading 'STREAM/00000.m2ts' at pos 0000000098b5d600

yippiekayee:

Naturally, I do not expect you to post the entire file, but if you have time, run these commands, and post the files:

dd if=00000.m2ts of=shablock1.bin bs=512 skip=5004011 count=1
dd if=00000.m2ts of=shablock2.bin bs=512 skip=7660563 count=1

yippiekayee

8th December 2008, 09:46

Here's MKB.RO.info from Prison Break. 00000.m2ts is the only episode on that disc and it it 11GB so posting is out of the question both for bandwidth as well as legal reasons.
@rumpan: Do you have PUO/advertising/region code/BD Live removal active in AnyDVD? I turned all that off so that the output should match the output of DumpHD (although I have yet to hash non m2ts files.. but the Bond movies TMT doesn't play will be hashed soon since my AnyDVD HD rip plays in folder mode whereas the DumpHD rip doesn't).

Several sites were reporting of a break of BD+ which never occurred and now the vendor of my player seems to have stopped releasing updates. Being 1-2 months behind SlySoft is acceptable for me. What makes you think that this won't happen again regardless of whether you beat Slysoft to the punch? Sensationalist reporting is a fact of life... don't you recall the news about AACS being broken? I'd turn this whole argument around and say that the longer the current code stays unbroken and the more titles that are being released which cannot be copied, the more it strenghtens Fox in their belief that they shouldn't just give up on BD+ because the cost outweighs the benefits. If you think back to early this year, it was rumored that Fox was close to dumping BD and one of the reasons was cost.. with BD+ they have to swallow higher cost than other studios which are content with AACS.

Accident

9th December 2008, 02:10

loo3aem3ON: Clarify some of the recent changes for me please. You send events up to 50 "files", is the 50 just arbitrary, or a known upper-limit? In terms of program flow, do you only ever receive one conv_tab, or in fact, handle receiving multiples, and merge them in? Currently libbluray only receives and parses exactly one conv_tab. (Convert binary format to C structures)

Rupan:
Comparing the two conv_tab.bin, I find this to be interesting:

000006e0 00 00 50 3d 00 00 63 34 00 00 66 4e 00 00 6d bb #Deb
000006e0 00 00 50 3d 00 00 63 34 00 00 6d bb 00 00 75 38 #lib

000006f0 00 00 75 38 00 00 8b af 00 00 8e 2e 00 00 9a a9 #Deb
000006f0 00 00 8b af 00 00 8e 2e 00 00 9a a9 00 00 da 4d #lib

Debugger has an extra 4-byte value in there, otherwise the files match quite well. And a few more, we are off by about 36bytes after that index. I must be removing too many Type-2 descriptors.

yippiekayee: Don't miss my sha request in my previous post :)

loo3aem3ON

9th December 2008, 03:36

loo3aem3ON: Clarify some of the recent changes for me please. You send events up to 50 "files", is the 50 just arbitrary, or a known upper-limit?
The 50 is arbitrary. The right way to do it would probably be to read the filenames from the BDVM/STREAM/ directory and then calculate the parameter for event #0110 with some unknown mapping f(filenmame) = parameter. It's also the possible that the player gets this mapping from the content code. Maybe after event #0110 with last parameter = 0xFFFF?

In terms of program flow, do you only ever receive one conv_tab, or in fact, handle receiving multiples, and merge them in?
With those events #0110 (last parameter going up from 1 to 50) i announce the playback of hopefully all BD+ damaged files to get all the different subtables of the conversion table. Some conversion tables are bigger than 4MB so there is no way for the content code to pass such a big table to the player with a single call of TRAP_GetConversionTable. So you have to handle multiple calls of that trap and finally merge all the different subtables you've got into a big conversion table. I do this by creating a subtable object for each new subtable and after decryption/filtering i build a conversion table from all objects.

I must be removing too many Type-2 descriptors.
Yes. The first byte of the first valid descriptor you've mistakenly eliminated is 0xB6.

yippiekayee: Don't miss my sha request in my previous post :)
You really should add support for this simple hash database format. If you can't open the file you query this database. Don't hesitate to ask me if you have any questions about it.

Rupan

9th December 2008, 03:42

loo3aem3ON: my apologies, but I don't think that I am qualified to participate in development here. I can code in C and Java just fine, but when it comes to math-based security I just get lost. I averaged a B in calculus I through III and hated every minute of it. FYI, I did try to read your BD+ documentation and failed to grasp it to a point where I can help.

Accident: its good that what work I can do is helping.

Given the above, I'd be glad to help with code that doesn't involve crypto so let me know what I can do.

Accident

9th December 2008, 06:35

The 50 is arbitrary. It's also the possible that the player gets this mapping from the content code. Maybe after event #0110 with last parameter = 0xFFFF?

Yes, I had noticed that some BD+ disks does do that call. If I find time, maybe I'll check to see if I happen to find the right number.

So you have to handle multiple calls of that trap and finally merge all the different subtables you've got into a big conversion table.

I suspected as much! I will modify the sources to handle being called multiple times, and either add-with-sort, or postpend, and sort later.

If a given title has multiple subtable, and you are only given one decrypt key, I assume you keep the key, or ask for it multiple times? Do we have the BDSVM where this is required that I can test with?

Yes. The first byte of the first valid descriptor you've mistakenly eliminated is 0xB6.

I thought everyone knew that 0xB6 is evil.

You really should add support for this simple hash database format. If you can't open the file you query this database. Don't hesitate to ask me if you have any questions about it.
Naturally, and it won't be a problem, but I didn't think we had hashdb file for PrisonBreak anyway, so I can't test it either way.

Checking through the code I find:

if (entry->flags & 0x80) {

Terrible! I shall fix this asap. I am definitely removing too many repair descriptors.

Edit:

Now I'm starting to second-guess myself. Technically the &0x80 test is wrong, since it would include "type 3" descriptors in what is being removed, but 0xB6 is a valid type 2 ? So it would then presumably be the bit check after...

bits = entry->flags & 0x3f; // 6 bits, 0-63
if (!(mask[ (bits >> 3) ] & (1<<(bits & 0x07)))) {

..that is incorrect.

Edit2: Comparing against Debugger, I see you mark them inactive if type is 0 and 3, and well as using 7-byteoffset. I have done the same sort of changes, and we should retry against DH4.

Edit3:

Rupan:

That is excellent, thank you for your assistance.

Rupan

9th December 2008, 11:49

The updates you pushed out in svn revision 159 fixed DH4. The conversion tables produced by libbluray and bdvmdbg are byte identical. Good work!

I've attached a 64-bit build to this post in case anyone wants to try it out.

yippiekayee

9th December 2008, 19:15

Don't miss my sha request in my previous post I didn't really get what you wanted but I attached the mkb_ro file to my post but for some reason it never showed up - and my post suddenly appears to be edited by a mod.
Either way.. I'm attaching the hash_db file created by the debugger and here I ran sha1sum on a few files:
3976563d407eb565a822dfdc5bfad4bb5c299eff 00000.m2ts
da9af84a1221f06b66178fd3d1c3cb8e35d93538 Content000.cer
ea9b1b1ba6bd7d3cdf8d03e146b0cb891dfa8bbb ContentHash000.tbl
ffdfb3d33001abaadded8ac0ea4a447b6347186f ContentRevocation.lst
5cf9723d9e870514adb31ae8f7cd1dd591c1fa5d CPSUnit00001.cci
28357c70a24094c8bdff90acfd886cec0f015dad MKB_RO.inf
285496166a6bad574f49916cb2da4edc8cdb90fd MKB_RW.inf

dirio49

9th December 2008, 20:10

@yippiekayee
the dd command is a linux command, to read in raw mode (unless i am mistaken)
this link is for the windows executable
http://www.chrysocome.net/downloads/dd-0.5.zip

site http://www.chrysocome.net/dd

hope this helps

loo3aem3ON

9th December 2008, 20:45

Manually extracting blocks from arbitrary files on the disc is no acceptable long term solution. You should use the hash database which is easy to implement and automatically generated.

Accident: You can find an example in the repository now. (direct link) (https://trac.assembla.com/bdplus/browser/trunk/doc/debugging/hash_db.txt)

Guest

9th December 2008, 21:54

I attached the mkb_ro file to my post but for some reason it never showed up - and my post suddenly appears to be edited by a mod. The file appeared to be a complete intact file from a DVD. We can't post copyrighted things like that.

Accident

10th December 2008, 01:12

Manually extracting blocks from arbitrary files on the disc is no acceptable long term solution. You should use the hash database which is easy to implement and automatically generated.

At the time, we had neither file format. "dd" just meant anyone could run it, on any OS, without needing to setup java, or run debugger. But, since he already had the hashdb.bin file, even better. I have grabbed it, and implementing hashdb support now.

Edit:

The hash_db.bin from yippiekayee, which I think is for PBS3D4, runs like this:

[diff] Attempting to open 'hash_db.bin' looking for 'AACS/MKB_RO.inf'
[diff] namehash: 00 00 00 00 00 00 00 00 00 00 02 00 41 41 43 53 2F 4D 4B 42 5F 52 4F 2E 69 6E 66
[diff] find hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] found digest, reading 00000014 (20) bytes...
[diff] Attempting to open 'hash_db.bin' looking for 'STREAM/00000.m2ts'
[diff] namehash: 00 00 00 00 E9 C8 26 00 00 00 02 00 53 54 52 45 41 4D 2F 30 30 30 30 30 2E 6D 32 74 73
[diff] find hashdb: 93 C4 D2 BB 20 66 51 56 AB C7 B4 DE 6C 19 58 F7 5A 29 0D 0F
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D
[diff] Attempting to open 'hash_db.bin' looking for 'STREAM/00000.m2ts'
[diff] namehash: 00 00 00 00 98 B5 D6 00 00 00 02 00 53 54 52 45 41 4D 2F 30 30 30 30 30 2E 6D 32 74 73
[diff] find hashdb: C7 B2 99 A3 91 7E 9C AF 2C D6 62 3C 07 A9 0C 6D 01 25 9C 33
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D

So it successfully finds the first entry, which makes me believe my hash_db.bin code is correct, but fails to find the other two. Hexdump confirms there are only 3 entries in that hash_db.bin.

loo3aem3ON: You may want to throw in a comment in hash_db.bin documentation not to SHA hash the C-string NULL terminator byte for the filename. I guess as much, but I did try both just to be safe. :) Although your examples clearly show you not including them.

Edit2:

Oh I see, some file-paths are inside of BDMV/ and some are full-path. Quite annoying that. However, now I get:

[diff] Attempting to open 'hash_db.bin' looking for 'AACS/MKB_RO.inf'
[diff] find hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] found digest, reading 00000014 (20) bytes...
[diff] Attempting to open 'hash_db.bin' looking for 'BDMV/STREAM/00000.m2ts'
[diff] find hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D
[diff] found digest, reading 00000014 (20) bytes...
[segment] Starting decode of conv_tab.bin: 0x4dfafc (1044836)
[segment] num tables 79

[segment] Key 0, 0: 9CF737C8A6E37D314472789794CC19A7 mask:326FC93CE78C0C6A

[segment] Key 0, 7: 20AB48B7B57FD04DE46E81855DCD8E65 mask:FFFFFFFFFFEFFFFF
[segment] removing entry 121 (flags 80: bits 20 => byte 5, set 10 to false)

MD5 (PBS3D4/conv_tab.bin) = 694e63c6e486231659c63619fa381b98

Which might just be the correct result.

loo3aem3ON

10th December 2008, 15:33

loo3aem3ON: You may want to throw in a comment in hash_db.bin documentation not to SHA hash the C-string NULL terminator byte for the filename. I guess as much, but I did try both just to be safe. :) Although your examples clearly show you not including them.

Edit2:

Oh I see, some file-paths are inside of BDMV/ and some are full-path. Quite annoying that. However, now I get:
The filename is exactly the string from the vm memory:RAP_MediaSHAFileHash(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst)
That is <FileNameLen> bytes from address <FileName>.

Nice to see you got it working. Do you also return <len'> ?

Given the above, I'd be glad to help with code that doesn't involve crypto so let me know what I can do.
You can improve the source code of the debugger. I don't have much experience programming in java. :rolleyes:
BD+ is basically an arbitrary program running on a virtual DLX like processor. This program executes some tests to check if it runs on a valid player (trusted environment) and if all tests passed it generates the conversion table (fix-up table) which is then used to repair the damaged m2ts files. Some tests involve trap calls which do cryptography but they make up only a very small part of the source code. ;)

Accident

11th December 2008, 04:57

Yep, I do set len', as in "the len of sha blocks we actually managed to read".

So, to return to the multiple conv_tab blocks, which BDSVM files has/needs this so I can test with it?

loo3aem3ON

11th December 2008, 20:36

Below is a development snapshot of the upcoming version 0.1.5 of the debugger. This version supports (diffarchive) snapshot recording. Instead of pressing "run" you need to select "snapshots" -> "Record snapshots" from the menu and then wait for about 30min (depending on your pc) for this process to complete. The snapshot archives are stored "snapshots".

BDVM debugger 0.1.5 (dev) (http://uploaded.to/?id=jf0qm6)

For those discs from which you couldn't get any conversion table with libblueray but the debugger seems to execute everything properly please record snapshots. Please upload the content of the "BDSVM directory from the disc as well as the contents of the "snapshots" directory the debugger created after recording.

Accident: I have switched everything (raw diff segments as well as the diffarchive metadata) to BIG endian as requested. All future snapshot packages will be big endian. I have already converted the jumper package to big endian. :)

So, to return to the multiple conv_tab blocks, which BDSVM files has/needs this so I can test with it?
Yippiekayee reported the problem in posting #506 (https://forum.doom9.org/showpost.php?p=1220969&postcount=506) which also contains the link to the content code. My reply in posting #511 (https://forum.doom9.org/showpost.php?p=1221095&postcount=511) gives a short explanation of the problem.

Accident

12th December 2008, 03:44

Accident: I have switched everything (raw diff segments as well as the diffarchive metadata) to BIG endian as requested. All future snapshot packages will be big endian. I have already converted the jumper package to big endian. :)

That was surprising, and thanks for the warning. I will keep that in mind when I next get snapshots.

Yippiekayee reported the problem in posting #506 (https://forum.doom9.org/showpost.php?p=1220969&postcount=506) which also contains the link to the content code.

The hash_db.bin file appears to be in the previous hash_db format. I don't suppose anyone has the file in the current format? Guess I could temporarily add support for the old format as well.

Edit: Not to worry, I have hardcoded it for now, so I can test it against multiple conv_tab entries.

Edit 2:

Like that of Debugger, I post 110,0,1 followed by 110,0,2 .... 110,0,24. I receive a non-zero conv_tab for both "fileNumber" 1, and 2. The rest result in 0 length conv_tab. This makes me think there are two parts of me to get. But what I can not quite figure out, is that both conv_tab that I receive are mostly the same. I save the data from 110,0,1 and 110,0,2, do a binary compare, and they are virtually identical. There are less than 8 bytes that differ (roughly in the middle of the file). I am unsure what sort of "merge" I should do with multiple conv_tables.

loo3aem3ON

12th December 2008, 09:30

Like that of Debugger, I post 110,0,1 followed by 110,0,2 .... 110,0,24. I receive a non-zero conv_tab for both "fileNumber" 1, and 2. The rest result in 0 length conv_tab. This makes me think there are two parts of me to get. But what I can not quite figure out, is that both conv_tab that I receive are mostly the same. I save the data from 110,0,1 and 110,0,2, do a binary compare, and they are virtually identical. There are less than 8 bytes that differ (roughly in the middle of the file). I am unsure what sort of "merge" I should do with multiple conv_tables.
I have never compared them and i am surprised they differ. If you happen to have both versions on disc could you please upload them so i can take a look? The way i handle it currently is creating a subtable object for every new subtable i get. If an object with that subtable id already exists i ignore it. Therefor in this example all subtables which come with the second TRAP_GetConversionTable call are ignored.

I really need to figure out the meaning of the second parameter of event#0110. My closest guess was that the m2ts are enumerated 1...[number of files]. Example:
filename | id
---------------------
00000.m2ts | 1
00002.m2ts | 2
00004.m2ts | 3
00006.m2ts | 4
00008.m2ts | 5
00010.m2ts | 6
00012.m2ts | 7
00014.m2ts | 8
00016.m2ts | 9
00018.m2ts | 10
.... | ...
So when you start the playback of 00012.m2ts you would create an event 0110,0,7. More tests are needed to confirm/disprove this.

Accident

12th December 2008, 09:43

Certainly. jumper_raw_tables.7z (http://uploaded.to/?id=us4cw5). These are saved on trap_ConvertionTable() so they have not been touched by me at all. Alas, this includes decoding.

The differences are:

00100180 00 00 00 00 00 03 b5 c0 00 06 24 10 00 3d fc c4 |......��..$..=��|
00100180 00 00 00 00 00 03 b5 f8 00 06 24 10 00 3d fc c4 |......��..$..=��|

00100200 00 06 29 8c 00 3d fd 38 00 02 e4 c4 00 03 b5 c0 |..)..=�8..��..��|
00100200 00 06 29 8c 00 3d fd 38 00 02 e4 c4 00 03 b5 f8 |..)..=�8..��..��|

00100250 00 02 50 f8 00 3d fd 78 00 00 00 00 00 00 00 10 |..P�.=�x........|
00100250 00 02 50 f8 00 3d fd 78 00 00 00 00 00 00 00 0b |..P�.=�x........|

00100260 00 00 00 70 33 92 f8 11 00 02 ea 10 00 1e ff fc |...p3.�...�...��|
00100260 00 00 00 87 39 07 9b c3 00 02 ea 10 00 1e ff fc |....9..�..�...��|

00100280 00 00 23 80 00 00 00 00 00 3d fd ac 00 02 ef 24 |..#......=�..�$|
00100280 00 00 23 cc 00 00 00 00 00 3d fd ac 00 02 ef 24 |..#�.....=�..�$|

I will leave my conv_tab code untouched for now. If I get one, that is the one I deal with.

I would have expected to use "tableID" somewhere as well (the tableID stored in conv_tab), but so far, I have not.

loo3aem3ON

12th December 2008, 17:37

Thanks.

Were you able to find out what TRAP_#8020 does? I would like to implement it. (see posting #507 (https://forum.doom9.org/showpost.php?p=1220977&postcount=507)).

I will leave my conv_tab code untouched for now. If I get one, that is the one I deal with.
The conversion table may be split. In theory you won't get the complete conversion table unless you announce the playback of all files. Please read again postings #461 (https://forum.doom9.org/showpost.php?p=1218518&postcount=461), #471 (https://forum.doom9.org/showpost.php?p=1218761&postcount=471) and #479 (https://forum.doom9.org/showpost.php?p=1218870&postcount=479). I'll investigate the mapping [file number] -> [second parameter of event#0110] soon. Currently i'm looking at the data returned by the callbacks/events.

yippiekayee

12th December 2008, 18:20

@Accident: are you working on the US version of Jumper? And since Rupan had better success with the latest code could you post the latest libbluray for windows?
I did not keep the md5sums or ripped files of my previous discs but I kept the convtable of every successful rip so once you think you're ready for some more testing I could run a few discs and give you the debugger and libbluray output for comparison.

For those discs from which you couldn't get any conversion table with libblueray but the debugger seems to execute everything properly please record snapshots. Please upload the content of the "BDSVM directory from the disc as well as the contents of the "snapshots" directory the debugger created after recording.I take it that's mean for me right? So for now I think I'm supposed to run this on Dr. No.
I'm afraid I'm not much of a math buff either.. I suffered through cryptography in college and was glad when it was over.. same story with the majority of other math classes I had to take.. I failed almost no classes but the three I did fail all were pure math (at least according to my definition.. some would argue it's applied math but to me it's all the same). I don't mind ripping discs or creating those snapshots to help out, but as far as attacking BD+ or making this thing work with the latest BD+ titles goes, I don't think I'm your guy.

I just got 4 new BD+ discs