Finally handling BD+ (?) [Archive]

gioowe

24th August 2008, 14:25

Now that MKBv7 is public I think it's time to concentrate on BD+. I can't imagine that SlySoft is the only one to manage that protection. So how do they do it? Magic? I don't think so! And obviously they didn't reverse engineer PowerDVD - this was the third time that AnyDVD handles BD+ but PowerDVD can't. PowerDVD users had to wait for an update. So, did they kill a player? Could be easier. Did anyone ever tried something like this? What's the current status of BD+?

Let's wind up and find a strategy. How can hardware players help? Has anyone any information about BD+ (beside that marketing crap from Macrovision)?

Only productive posts welcome.

Oopho2ei

24th August 2008, 15:15

Could be an implementation problem in powerdvd. For example you could accidentally do if (some_array[123] == 0x01) {...} and some_array is an array of integers (32 bit). You were supposed to compare one BYTE of the array and not 4. As long as there are enough zeros following the 01 you will never notice the bug. For some discs that is not true so they fail to playback because of that bug. Normally you will notice such mistakes when you reverse engineer the code.

Is there a list of BD+ discs so i could choose which one to buy? :rolleyes:

Edit: here are some:
28 Days Later
28 Weeks Later
August Rush
Cast Away
Dan in Real Life
Die Hard 1-4
Dogma
Fantastic 4 ROSS
From Hell (Region A)
Gattaca
Hitman
I Robot
Independence Day
Man on Fire
Master and Commander
Mr & Mrs Smith
Mr Brooks
No Country For Old Men
Rescue Dawn
Robocop
Ronin
Sleuth
Sunshine
The Day After Tomorrow
The Simpsons Movie

and almost all recent releases: http://bluray.highdefdigest.com/releasedates_historical.html

gioowe

24th August 2008, 17:04

I doubt that PowerDVD can implement their own BD+ code - they surely get a library from Macrovision. Closed source. Everything else would be extremely stupid. So the bug is in Macrovision's library. That would encourage my opinion that SlySoft has something in the loop with hardware players.

Peer van Heuen

24th August 2008, 17:19

I doubt that PowerDVD can implement their own BD+ code - they surely get a library from Macrovision. Closed source. Everything else would be extremely stupid. So the bug is in Macrovision's library. That would encourage my opinion that SlySoft has something in the loop with hardware players.

I'll just say: due to certain properties of BD+, once you're past a certain point, you can handle it pretty much without reversing - BD+ itself then helps you out - on any player ;)

Oopho2ei

24th August 2008, 17:23

I doubt that PowerDVD can implement their own BD+ code - they surely get a library from Macrovision. Closed source. Everything else would be extremely stupid. So the bug is in Macrovision's library. That would encourage my opinion that SlySoft has something in the loop with hardware players.
What makes you think the bug is inside that "library"? It could even be a mastering problem and powerdvd is simply following the specification too closely. Why do you explicitly point at powerdvd? What about the other software players?

gioowe

24th August 2008, 18:07

What makes you think the bug is inside that "library"? It could even be a mastering problem and powerdvd is simply following the specification too closely. Why do you explicitly point at powerdvd? What about the other software players?

What other players? WinDVD? ArcSoft? Or Nero?

Anyway, if AnyDVD can play all BD+ and PowerDVD requires an update - that's something very, very interesting.

gioowe

24th August 2008, 18:10

I'll just say: due to certain properties of BD+, once you're past a certain point, you can handle it pretty much without reversing - BD+ itself then helps you out - on any player ;)

That sounds interesting. So do you say that BD+ can "updates" itself or for instance a player helps out?

Oopho2ei

24th August 2008, 18:19

I am guessing he means that because bd+ enforces a certain memory footprint (e.g. keys, video etc have to be at a certain fix virtual memory address) he can use a memory dump of the virtual machine interpreter and include it in anydvd as precompiled code module. He would only have to make sure the remaining data vm opcode/keys/video content is at the correct address.

Peer van Heuen

24th August 2008, 19:44

I am guessing he means that because bd+ enforces a certain memory footprint (e.g. keys, video etc have to be at a certain fix virtual memory address) he can use a memory dump of the virtual machine interpreter and include it in anydvd as precompiled code module. He would only have to make sure the remaining data vm opcode/keys/video content is at the correct address.

No, that's not it (I thought we were past that silly memory dumping era anyway, that only works for badly written security code :) ). This foot-/fingerprint stuff is only a very minor part of BD+ and what you're saying would be rather trivial.

Actually you'd have to know how BD+ really works, to know what I meant (and even then you probably wouldn't ).
But if I start unraveling that, I'd be finding myself looking for a new job by next week :)

So, I'll better shut up from this point on - should have done that in the first place, sorry... :o

gioowe

24th August 2008, 19:49

I am guessing he means that because bd+ enforces a certain memory footprint (e.g. keys, video etc have to be at a certain fix virtual memory address) he can use a memory dump of the virtual machine interpreter and include it in anydvd as precompiled code module. He would only have to make sure the remaining data vm opcode/keys/video content is at the correct address.

I don't think that. Or you can never create new hardware players or software players that can decode the first released BD+ disks. And if e.g. each certain data has to be at a specific location for all known and future player then there's no point. That's not a footprint.

Oopho2ei

24th August 2008, 20:16

I don't think that. Or you can never create new hardware players or software players that can decode the first released BD+ disks. And if e.g. each certain data has to be at a specific location for all known and future player then there's no point. That's not a footprint.
To be honest i don't understand what you are trying to say. Also i can't follow the way you try to argue that slysoft reverse engineered a hardware player instead of a specific software player to uncover the secrets of BD+. The code which is run on the virtual machine needs to be able to interfere with the decryption process and therefor BD+ imposes some restrictions on the way the memory is organized. But as Peer pointed out in his previous posting those restrictions are rather small.

No, that's not it (I thought we were past that silly memory dumping era anyway, that only works for badly written security code :) ). This foot-/fingerprint stuff is only a very minor part of BD+ and what you're saying would be rather trivial.
It was the best i could think of with my very limited knowledge about BD+. For an open source implementation the VM would have to be fully reverse engineered anyway. I will get a BD+ movie and see what the players do.

Actually you'd have to know how BD+ really works, to know what I meant (and even then you probably wouldn't ).
But if I start unraveling that, I'd be finding myself looking for a new job by next week :)

So, I'll better shut up from this point on - should have done that in the first place, sorry... :o
Thank you for your help.

Oopho2ei

24th August 2008, 20:28

I made some notes of what i was able to find out about BD+ browsing the web. Maybe this helps to clear things up and avoid misunderstandings:
BD+ involves the player running a small virtual machine whenever a BD+ disc is inserted. The virtual machine is a small BD+ content code interpreter that includes 100 lines of code and 60 different instructions. The VM loads additional code from the Blu-ray disc, which can perform one of three functions:

- Transform code, which is used to correct sections of video data which are deliberately corrupted on the video stream part of the disc. If hackers strip out only the video code itself, it will display corrupted sections without the application of transform code. This code may also be used to add digital watermarks to certain discs, which can then be used for forensic investigations of leaked copyrighted material.
- Basic countermeasures, which can check a player that is known to have had its hardware hacked for example, a patch to the drive's firmware and detect and respond to this hack. This allows new discs to disable playback on standalone players that have been hacked, without having to revoke the license keys of the entire model or line of players.
- Advanced countermeasures, which can load native code that runs directly on the Blu-ray player during the playback process. This code can do any number of things, but is designed to provide additional methods of protection should the basic countermeasures fail.

Once the disc is removed from the player, the virtual machine unloads and is no longer present in memory. This returns the Blu-ray player to its initial state, so that even if a machine fails to play a BD+ title, it should still (again, in theory) play previously-released Blu-ray titles that do not have BD+ protection.

According to the Blu-ray Disc Association (BDA), for hackers to successfully attack the BD+ system they would have to first extract the AACS keys (which has already been done quite successfully) and then overcome title-specific security code by reverse-engineering the BD+ virtual machine. While this last bit is definitely an additional challenge, it is by no means impossible.

The announcement of the BD+ spec means that some studios that have been holding off on releasing high-definition content may now be more willing to release titles: it was rumored that Fox Studios had been delaying titles because of the previous AACS cracks. The availability of more titles on Blu-ray than HD DVD (since the latter does not support BD+) could swing the format wars even farther in Blu-ray's direction, especially when combined with events such as the Blockbuster announcement.

More worrying than a resolution of the high-definition format wars, however, is what studios might want to do with the additional powers that BD+ provides them. The ability to run any sort of code in the name of "advanced countermeasures" also brings the power to limit content by other means: timed-release and expiring discs are just some of the possibilities.
If you have any further (useful) information please feel free to post them here.

Oopho2ei

25th August 2008, 19:34

I have had a look at the difference of the output of dumphd and anydvd for a bd+ movie. Using vbindiff i compared the 00001.m2ts:
- 072D4764: C3 7C 9C 9C 95
+ 072D4764: AA 2D 40 8E 4A

- 072D85F1: 0A 0E BC E9 99
+ 072D85F1: 4D EF 4B 6B E5

- 0736ED6A: FB 51 33 7A 48
+ 0736ED6A: B6 F7 78 EC CB

- 0736FA18: 2F 03 43 31 2E
+ 0736FA18: 7E 61 A3 C6 C1

- 073C83B4: EC E9 4D A4 39
+ 073C83B4: 06 F3 AA 7C 59
...
It seems to be always 5 bytes which have to been replaced by the virtual machine. Has anyone had a look at this? Please post your findings.

chavonbravo

26th August 2008, 05:42

Is there a certain file on bd+ discs that contains the bd+ code? How does the vm in hw/sw player extract the information that the disc requires to play?

Edit:
found better explanations for the vm:

http://securityevaluators.com/pdf/spdc_aacs_2005.pdf
http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070033419%22.PGNR.&OS=DN/20070033419&RS=DN/20070033419

That last one is a gold mine of information about the vm instructions used. Wish I had a bd+ disc to test with.

Revgen

26th August 2008, 06:37

due to certain properties of BD+, once you're past a certain point, you can handle it pretty much without reversing

Sounds like what I had to go through with my GF. :D

Oopho2ei

26th August 2008, 18:29

found better explanations for the vm:
http://securityevaluators.com/pdf/spdc_aacs_2005.pdf
http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070033419%22.PGNR.&OS=DN/20070033419&RS=DN/20070033419
That last one is a gold mine of information about the vm instructions used.
Thank you! That looks really interesting!

Is there a certain file on bd+ discs that contains the bd+ code? How does the vm in hw/sw player extract the information that the disc requires to play?
There is a directory "BDSVM" (at the same level as aacs) containing the files "00000.svm, 00001.svm and 00001.svm". In the subdirectory "backup" there is a copy of those files.

Wish I had a bd+ disc to test with.
After the long header the data looks very random. So it is possibly encrypted (with the media key mabe?). I currently try to figure this out.

Sounds like what I had to go through with my GF. :DWell if you know the instruction set you can simply attach a compatible debugger to her brain when she misbehaves to see what is going on in her mind. :D Maybe peer was trying to tell us that the virtual machine is running a well known (extended) instruction set. :rolleyes:

Btw the DLX processor architecture is very primitive. Usually it is being used in university to demonstrate some basic principles of processor architecture as vhdl/verilog code.

Oopho2ei

26th August 2008, 21:51

A note about the files stored in the BDSVM directory: 00000.svm, 00001.svm and 00002.svm. They all have a size of exactly 20MB and each file consists of exactly 10 sections. Every section has the same size of 2MB.

The first section starts with:
0000:0000 42 44 53 56 4d 5f 43 43 01 00 00 00 01 07 d7 0b
0000:0010 06 00 00 00 00 07 ec e4
followed by:
b3 ff ff fc 00 00 ... lots of zeros .. some data ... zeros

Every other section starts with:
XXX0:0010 42 44 53 56 4d 5f 43 43 01 00 00 00 01 07 d7 0b
XXX0:0010 06 00 00 00 00 1f ff c0
followed by random data.

The beginning of the last file (00002.svm) has a visible pattern (vector table?):
0000:0000 42 44 53 56 4d 5f 43 43 01 00 00 00 01 07 d7 0b
0000:0010 06 00 00 00 00 1f ff c0 00 24 00 00 00 01 01 53
0000:0020 00 00 05 54 00 00 05 58 00 00 0f 84 00 00 1f c8
0000:0030 00 00 2e 54 00 00 3c f4 00 00 4b 94 00 00 59 80
0000:0040 00 00 69 4c 00 00 78 c8 00 00 84 ac 00 00 91 1c
0000:0050 00 00 9c ec 00 00 a8 80 00 00 b4 3c 00 00 c0 5c
0000:0060 00 00 cc 40 00 00 d7 20 00 00 e2 b4 00 00 ee 84
0000:0070 00 00 f8 24 00 01 02 28 00 01 0e e8 00 01 1c c0
0000:0080 00 01 29 44 00 01 35 00 00 01 40 bc 00 01 4c c8
0000:0090 00 01 5a 78 00 01 66 48 00 01 72 cc 00 01 7f 28
0000:00a0 00 01 8a 80 00 01 95 38 00 01 a2 0c 00 01 ae 68
0000:00b0 00 01 bc 2c 00 01 ca 68 00 01 d7 dc 00 01 e4 10
0000:00c0 00 01 f1 70 00 01 fe 1c 00 02 0a 78 00 02 17 38
0000:00d0 00 02 24 20 00 02 32 34 00 02 3d 78 00 02 49 fc
0000:00e0 00 02 56 6c 00 02 60 ac 00 02 6b f0 00 02 79 28
0000:00f0 00 02 85 70 00 02 92 6c 00 02 9f 18 00 02 ac f0
0000:0100 00 02 b9 74 00 02 c5 58 00 02 d1 00 00 02 da 14
0000:0110 00 02 e5 bc 00 02 ef c0 00 02 fb 40 00 03 05 08
0000:0120 00 03 13 94 00 03 20 68 00 03 2b ac 00 03 36 8c
0000:0130 00 03 45 7c 00 03 52 14 00 03 5f 24 00 03 6a e0
0000:0140 00 03 77 f0 00 03 86 7c 00 03 93 78 00 03 9f 0c
0000:0150 00 03 ad 34 00 03 b8 3c 00 03 c3 1c 00 03 cc bc
0000:0160 00 03 da a8 00 03 e4 48 00 03 ed e8 00 03 f9 68
0000:0170 00 04 08 d0 00 04 15 04 00 04 20 ac 00 04 2d 30
0000:0180 00 04 37 d4 00 04 44 6c 00 04 50 dc 00 04 5e 28
0000:0190 00 04 6c 8c 00 04 7a dc 00 04 88 c8 00 04 94 5c
0000:01a0 00 04 a0 2c 00 04 ac 24 00 04 b7 f4 00 04 c2 70
0000:01b0 00 04 ce e0 00 04 da d8 00 04 e8 74 00 04 f5 d4
0000:01c0 00 05 01 e0 00 05 0f 40 00 05 1c 28 00 05 26 90
0000:01d0 00 05 31 c0 00 05 3d 7c 00 05 49 c4 00 05 56 c0
0000:01e0 00 05 61 dc 00 05 6d 84 00 05 7b fc 00 05 86 a0
0000:01f0 00 05 93 38 00 05 9d 8c 00 05 a8 e4 00 05 b2 d4
0000:0200 00 05 bf 30 00 05 cb 50 00 05 d6 1c 00 05 df 80
0000:0210 00 05 ea 10 00 05 f7 84 00 06 02 3c 00 06 0f 88
0000:0220 00 06 1c 84 00 06 26 b0 00 06 32 6c 00 06 3e c8
0000:0230 00 06 4d cc 00 06 5b 54 00 06 68 a0 00 06 73 08
0000:0240 00 06 7d c0 00 06 89 18 00 06 94 20 00 06 9d d4
0000:0250 00 06 a9 90 00 06 b3 08 00 06 bd 5c 00 06 c8 78
...
The following section is a copy of this section. All other sections look random. I don't see any pattern. Btw. "42 44 53 56 4d 5f 43 43" = "BDSVM_CC". I have seen a player parsing the first section by first verifying this string and then comparing [section offset+0x14] with 0x001FFFC0. Note that the next section starts at current section offset+0x00200000.

Edit: If your section headers look different please post them! :)

gioowe

27th August 2008, 16:09

Mine look the same. I think I read something about a 2 MB memory address space for the VM.

Oopho2ei

27th August 2008, 23:13

I have looked at section 2 of 000001.svm. It is decrypted with some sort of stream cipher. I have uploaded the first 64 kb of the decrypted section payload (skipping the 0x17 header bytes): http://uploaded.to/?id=59db0f
Note that the byte order for every dword has be reversed before application (xor) of the bitstream from the stream cipher (big endian <--> little endian conversion). So 0000:0000 c4 4e e6 65 00 01 00 00 02 00 00 00
is probably 0000:0000 65 e6 4e c4 00 00 01 00 00 00 00 02

I will look at the decryption algorithm soon.

Oopho2ei

29th August 2008, 20:46

The VM is at least involved (if not doing it all alone) in the decryption of those sections.

An typical instruction I (32 bit) is decoded like that:

Instruction I: C R[X], R[Y], R[Z]
C = I >> 0x1A
X = ((I >> 0x13) & 0x7C) >> 2
Y = ((I >> 0x0E) & 0x7C) >> 2
Z = ((I >> 0x09) & 0x7C) >> 2
C is the command number (6 bit so 64 possible commands) and X,Y,Z are register numbers (in this case). (>> means 'shift right').

The machine:
- 32 registers each 32 bit long
- probably big endian
- 4MB address space (first 128 bytes are register?)

instruction/command decoding:
cmd00: NOP
cmd01: ADD R[X],R[Y],R[Z] -> R[X] = R[Y] + R[Z]
cmd02: SUB R[X],R[Y],R[Z] -> R[X] = R[Y] - R[Z]
cmd03: MUL R[X],R[Y],R[Z] -> R[X] = R[Y] * R[Z]
cmd04: IDIV R[X],R[Y],R[Z] -> R[X] = unsigned(R[Y]) / R[Z]
cmd05: DIV R[X],R[Y],R[Z] -> R[X] = signed(R[Y]) / R[Z]
cmd06: SHL R[X],R[Y],R[Z] -> R[X] = R[Y] << (R[Z] & 0x1F)
cmd07: SHR R[X],R[Y],R[Z] -> R[X] = R[Y] >> (R[Z] & 0x1F)
cmd08: SAR R[X],R[Y],R[Z] -> R[X] = signed(R[Y]) >> (R[Z] & 0x1F)
cmd09: AND R[X],R[Y],R[Z] -> R[X] = R[Y] & R[Z]
cmd0A: OR R[X],R[Y],R[Z] -> R[X] = R[Y] | R[Z]
cmd0B: XOR R[X],R[Y],R[Z] -> R[X] = R[Y] ^ R[Z]
cmd0C: EQU R[X],R[Y],R[Z] -> if (R[Y] == R[Z]) R[X] = 1 else R[X] = 0
cmd0D: NEQU R[X],R[Y],R[Z] -> if (R[Y] != R[Z]) R[X] = 1 else R[X] = 0
cmd0E: LESS R[X],R[Y],R[Z] -> if (signed(R[Y]) < signed(R[Z])) R[X] = 1 else R[X] = 0
cmd0F: BLW R[X],R[Y],R[Z] -> if (unsigned(R[Y]) < unsigned(R[Z])) R[X] = 1 else R[X] = 0
cmd10: GRE R[X],R[Y],R[Z] -> if (signed(R[Y]) > signed(R[Z])) R[X] = 1 else R[X] = 0
cmd11: ABV R[X],R[Y],R[Z] -> if (unsigned(R[Y]) > unsigned(R[Z])) R[X] = 1 else R[X] = 0
cmd12: LEQ R[X],R[Y],R[Z] -> if (signed(R[Y]) <= signed(R[Z])) R[X] = 1 else R[X] = 0
cmd13: BEQ R[X],R[Y],R[Z] -> if (unsigned(R[Y]) <= unsigned(R[Z])) R[X] = 1 else R[X] = 0
cmd14: GEQ R[X],R[Y],R[Z] -> if (signed(R[Y]) >= signed(R[Z])) R[X] = 1 else R[X] = 0
cmd15: AEQ R[X],R[Y],R[Z] -> if (unsigned(R[Y]) >= unsigned(R[Z])) R[X] = 1 else R[X] = 0
cmd16: JMP R[Y] -> PC = R[Y]
cmd17: CALL R[Y] -> R[0x1F] = PC; PC = R[Y]
cmd18: ADDE R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); R[X] := R[Y] + C
cmd19: ADD R[X],R[Y],C -> C = I & 0xFFFF; R[X] := R[Y] + C
cmd1A: SUBE R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); R[X] := R[Y] - C
cmd1B: SUB R[X],R[Y],C -> C = I & 0xFFFF; R[X] := R[Y] - C
cmd1C: SHL R[X],R[Y],C -> C = I & 0x1F; R[X] = unsigned(R[Y]) << C
cmd1D: SHR R[X],R[Y],C -> C = I & 0x1F; R[X] = unsigned(R[Y]) >> C
cmd1E: SAR R[X],R[Y],C -> C = I & 0x1F; R[X] = signed(R[Y]) >> C
cmd1F: AND R[X],R[Y],C -> C = I & 0xFFFF; R[X] = R[Y] & C
cmd20: OR R[X],R[Y],C -> C = I & 0xFFFF; R[X] := R[Y] | C
cmd21: XOR R[X],R[Y],C -> C = I & 0xFFFF; R[X] := R[Y] ^ C
cmd22: EQU R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (R[Y] == C) R[X] = 1 else R[X] = 0
cmd23: NEQU R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (R[Y] != C) R[X] = 1 else R[X] = 0
cmd24: GRE R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (signed(C) > signed(R[Y])) R[X] = 1 else R[X] = 0
cmd25: ABV R[X],R[Y],C -> C = I & 0xFFFF; if (unsigned(C) > unsigned(R[Y])) R[X] = 1 else R[X] = 0
cmd26: LESS R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (signed(C) < signed(R[Y])) R[X] = 1 else R[X] = 0
cmd27: BLW R[X],R[Y],C -> C = I & 0xFFFF; if (unsigned(t0) < unsigned(R[Y])) R[X] = 1 else R[X] = 0
cmd28: GEQ R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (signed(C) >= signed(R[Y])) R[X] = 1 else R[X] = 0
cmd29: AEQ R[X],R[Y],C -> C = I & 0xFFFF; if (unsigned(C) >= unsigned(R[Y])) R[X] = 1 else R[X] = 0
cmd2A: LEQ R[X],R[Y],C -> C = sign_extend(I & 0xFFFF); if (signed(C) <= signed(R[Y])) R[X] = 1 else R[X] = 0
cmd2B: BEQ R[X],R[Y],C -> C = I & 0xFFFF; if (unsigned(C) <= unsigned(R[Y])) R[X] = 1 else R[X] = 0
cmd2C: JMP C -> C = PC + signed(unsigned(I) << 6) >> 6; PC = C
cmd2D: CALL C -> C = PC + signed(unsigned(I) << 6) >> 6; R[0x1F] = PC; PC = C
cmd2E: JZ R[Y], C -> C = PC + sign_extend(I & 0xFFFF); if (R[Y] == 0) PC = C
cmd2F: JNZ R[Y], C -> C = PC + sign_extend(I & 0xFFFF); if (R[Y] != 0) PC = C
cmd30: STR R[X] -> R[X] = I << 0x10
cmd31: LBSE R[X],R[Y] -> R[X] = sign_extended(mem[((R[Y] + sign_extended(I & 0xFFFF)) & 0x3FFFFF) ^ 0x03] & 0xFF)
cmd32: LB R[X],R[Y] -> R[X] = mem[((R[Y] + sign_extended(I & 0xFFFF)) & 0x3FFFFF) ^ 0x03] & 0xFF
cmd33: LWSE R[X],R[Y] -> R[X] = sign_extended(mem[((R[Y] + sign_extended(I & 0xFFFF)) & 0x3FFFFE) ^ 0x02] & 0xFFFF)
cmd34: LW R[X],R[Y] -> R[X] = mem[((R[Y] + sign_extended(I & 0xFFFF)) & 0x3FFFFE) ^ 0x02] & 0xFFFF)
cmd35: LDW R[X],R[Y] -> R[X] = mem[(R[Y] + sign_extended(I & 0xFFFF)) & 0x3FFFFC]
cmd36: MOVB R[X],[R[Y]+C] -> C = sign_extended(I & 0xFFFF); mem[((R[Y]+C) & 0x3FFFFF) ^ 0x03] = (unsigned char) R[X]
cmd37: MOVW R[X],[R[Y]+C] -> C = sign_extended(I & 0xFFFF); mem[((R[Y]+C) & 0x3FFFFE) ^ 0x02] = (unsigned int16) R[X]
cmd38: MOVDW R[X],[R[Y]+C] -> C = sign_extended(I & 0xFFFF); mem[(R[Y]+C) & 0x3FFFFC] = (unsigned int32) R[X]
cmd39: TRAP C -> trap[immediate]
cmd3A: SWDC R[Y] -> watch_cat_counter = R[Y];
cmd3B-cmd3F: NOP

Notation:
PC = program counter
C = a constant (part of instruction word)
I = the instruction word (uint32)
watchcat_counter = some counter which is set by cmd3A and decremented by every instruction by a command specific value. As long as the counter is > 0 program execution continues.
mem[] = the virtual machine memory
R[X] = register number X (uint32 R[32])
sign_extend(V) = MSB(V) ? 0xFFFF0000 | V : V;
MSB() = get the most significant bit
signed() = make it a signed integer (int32)
unsigned() = make it an unsigned integer (uint32)

Note that by the time the command handler is called the program counter (pc) has already been incremented by 4 (pointing at the next instruction).

It would be great if someone could start implementing the command interpreter (open source please). Please feel free to ask for more details. I will post the purpose of the remaining commands soon.

Oopho2ei

30th August 2008, 11:59

I have added some more commands. It is becoming increasingly difficult to find good names eg. BEQ (for 'below or equal' inintel terms) is usually used for 'branch if equal'. If you can think of better names/abbreviations you are also welcome to post your ideas. :)

Oopho2ei

31st August 2008, 02:39

I can see that things are getting more complicated and my C-like expressions become quite ugly. A lot of people are following this thread but no one dares to say something. So i decided to give a very small example of how this works.

You have probably seen the BDSVM folder on your blue rays with BD+ protection. Now open the 00000.svm in a hex editor (like khexedit) and scroll down to address around 0x1000. You are looking now at some of the instructions which are executed by the virtual machine at the very beginning.

...
0000:1000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:1010 00 00 00 00 00 00 00 00 b0 00 00 40 e0 3d 00 08
0000:1020 e3 fd 00 04 d4 20 00 00 e0 1d 00 00 d7 e0 11 54
0000:1030 e3 e0 11 50 b4 01 9b 0c d7 e0 11 58 e3 e0 11 50
0000:1040 e0 20 00 04 d7 fd 00 04 d4 3d 00 08 67 bd 00 1c
0000:1050 e4 00 00 10 b3 ff ff fc 00 00 00 00 bc 00 ff fc
0000:1060 bc 01 ff fc bc 02 ff fc bc 03 ff fc bc 04 ff fc
0000:1070 bc 05 ff fc bc 06 ff fc bc 07 ff fc bc 08 ff fc
0000:1080 bc 09 ff fc bc 0a ff fc bc 0b ff fc bc 0c ff fc
0000:1090 bc 0d ff fc bc 0e ff fc bc 0f ff fc bc 10 ff fc
0000:10a0 bc 11 ff fc bc 12 ff fc bc 13 ff fc bc 14 ff fc
0000:10b0 bc 15 ff fc bc 16 ff fc bc 17 ff fc bc 18 ff fc
0000:10c0 bc 19 ff fc bc 1a ff fc bc 1b ff fc bc 1c ff fc
0000:10d0 bc 1d ff fc bc 1e ff fc bc 1f ff fc c3 a0 00 00
0000:10e0 67 bd 11 5c c3 80 00 00 67 9c 10 00 d7 bd 00 00
...

Now let's see how this works:

1. Fetch: Execution starts with PC = 0x1000. Because we have a 0x18 byte long header this translates to the start address 0x1000+0x18=0x1018. If we look at the section from 00000.svm posted above we see our first instruction: "b0 00 00 40".

2. Decode: If we look at the highest 6 bits of the leftmost byte of this instruction. The binary represenation of 0xB0 = 0b10110000. The highest 6 bits are 0b101100 which is 0x2C. (Instead of converting it binary representation and back we could have simply divided by 4: 0xB0/4 = 0x2C.) Now if we lookup 0x2C in the table i posted above you will see that this is the encoding of the command "JMP C".

3. Execute: Now lets look at what this instruction does. The corresponding entry of the table says: "C = PC + signed(unsigned(I) << 6) >> 6; PC = C". So we first calculate a constant C = PC + signed(unsigned(I) << 6) >> 6. What the second part does is simply setting the highest 6 bits of our instruction I = 0xB0000040 to the value of the 7th highest bit. This is done here by first doing a logic shift left and then an arithmetic shift right. (The expressions "signed" and "unsigned" are used to distinguish between logic and arithmetic shift because C handles the shift operator differently of signed and unsigned values.) Now in our case the 7th highest bit is 0 so we can just overwrite the highest 6 bits with zeros. The result is C = PC + 0x40.
Now when the command dispatcher runs it already increments the value of the program counter (pc) by 4 (4 bytes is the length of every instruction) so that it is pointing at the next instruction even before the command handler (in this case for "jmp c") is executed. So in this case our program counter is PC = 0x1000 + 4 = 0x1004. Now according to the description "PC = PC + signed(unsigned(I) << 6) >> 6" our new program counter is PC = 0x1004 + 0x40 = 0x1044.

4. Write back: PC = 0x1044

Now lets look at the next instruction.

1. Fetch: At 0x1044+0x18 we have "bc 00 ff fc"

2. Decode: 0xBC / 4 = 2F so we have a "JNZ R[X],C". Looking in the instruction table we find the description "C = PC + sign_extend(I & 0xFFFF); if (R[Y] != 0) PC = C".

3. We first calculate C. Therefor we overwrite the 16 highest bits with the value of the 17th highest bit (this is what this expression simply does). The 17th highest bit is 1 so we result is C = PC + 0xFFFFFFFC = 0x1044. Now we see that R[Y] is checked for being different from zero. According to the description Y = ((I >> 0x0E) & 0x7C) >> 2. This expression reads the 5 bits directly following the command number (highest 6 bits) from the instruction. In our case this is Y = 0 so we have to check register 0 (R[0]) for being zero. In this case it is really zero because the virtual machine has just been started and we haven't manipulated anything except the program counter so far. Consequently "PC = C" is not being executed and we can forget C and proceed with the next instruction.

4. Write back: nothing....

Execution proceeds at 0x1048 ...

You will find more details scattered across the postings of this thread. Feel free to ask if you have questions. Explaining things to other people really helps finding bugs. :)

bmnot

31st August 2008, 13:05

Hi Oopho2ei,

I've been following the threads dealing with HDDVD/BluRay-Decryption on doom9 since the beginning and I must say you're doing an truly amazing job – despite the limited interest and resonance lately...
As probably many other members, I really appreciate your work on this topic (one reason is that there's no AnyDVD HD for my preferred operating systems...). :thanks:

Regarding the topic...
As already stated somewhere in the SlySoft forums, the BD+ virtual machine (obviously) has not much in common with Java (no stack operands, no objects). It looks more like some simple MIPS dialect.

Therefore - up to this point - it's pretty straightforward to implement an interpreter. Actually, you already more or less posted the source code...

It appears, there are at least three types of instructions (just another notation of what was already posted):

Type1 [6bit command][5bit Operand X][5bit Operand Y][5bit Operand Z][11 bit unused]
Type2 [6bit command][5bit Operand X][5bit Operand Y][16bit constant] (5bit constant in shift operations)
Type3 [6bit command][26bit constant]

Regarding the functionality of commands 36-3A, there are still operations missing to store values in memory, to perform systems calls and to execute native code.

It would be interesting to know how the VM deals with arithmetic overflows (ADD, MUL)... or if they are simply not allowed in BD+.

Is the PC counted in bytes or words? In the below implementation it's bytes (as in your tutorial). However, this would unnecessarily restrict the range of jump commands.

Below is a simple, experimental C++ implementaion of the BD+ VM based on your specs (compiles, but not tested). Targets were keeping it short, maintainable and close to your notation. Hope I got all the signs right... Not sure about the commands for loading values from memory...

/* this code is public domain */

// the following lines could be moved to a separate header file later
typedef unsigned char byte; // for less typing
typedef unsigned int uint; // dito

class ExperimentalBDVM {
public:
void runProgram(byte* pcBaseAddress, uint pcStartOffset = 0x1000);
};
// end header

const int MEMSIZE = 0x400000; // 4MB address space

uint makeint(byte b3, byte b2, byte b1, byte b0) {
return ((int)b3) << 24 | ((int)b3) << 16 | ((int)b2) << 8 | ((int)b0);
}

int _signed(uint x) { return (int)x; }
int _signed(int x) { return x; }
int _unsigned(uint x) { return x; }
int _unsigned(int x) { return (uint)x; }

void ExperimentalBDVM::runProgram(byte* pcBaseAddress, uint pcStartOffset) {

// init VM
uint PC = pcStartOffset; // program counter
uint R[32]; // registers
byte mem[MEMSIZE]; // memory

// init memory
for (int i = 0; i < MEMSIZE; i++) {
mem[i] = 0;
}

// temporary variables
byte* addr;
int b = 0;

// process commands in (currently infinite) lo
while(true) {

// instruction fetch
byte* pca = pcBaseAddress + PC; // PC memory address
uint I = makeint(pca[3], pca[2], pca[1], pca[0]); // instruction
uint C = I >> 0x1A; // command
uint X = ((I >> 0x13) & 0x7C) >> 2; // operand X
uint Y = ((I >> 0x0E) & 0x7C) >> 2; // operand Y
uint Z = ((I >> 0x09) & 0x7C) >> 2; // operand Z
uint C16 = I & 0xFFFF; // 16 bit constant
int C16S = _signed(_unsigned(I) << 16) >> 16; // same signed
uint C5 = I & 0x1F; // 5 bit constant
//uint C26 = I & 0x3FFFFFF; // 26 bit constant
int C26S = _signed(_unsigned(I) << 6) >> 6; // same signed
PC += 4; // already increase PC

// execute instruction
switch (C) {
case 0x01: R[X] = R[Y] + R[Z]; break; // ADD
case 0x02: R[X] = R[Y] - R[Z]; break; // SUB
case 0x03: R[X] = R[Y] * R[Z]; break; // MUL
case 0x04: R[X] = _unsigned(R[Y]) / R[Z]; break; // IDIV
case 0x05: R[X] = _signed(R[Y]) / R[Z]; // DIV
case 0x06: R[X] = R[Y] << (R[Z] & 0x1F); break; // SHL
case 0x07: R[X] = R[Y] >> (R[Z] & 0x1F); break; // SHR
case 0x08: R[X] = _signed(R[Y]) >> (R[Z] & 0x1F); break; // SAR
case 0x09: R[X] = R[Y] & R[Z]; break; // AND
case 0x0A: R[X] = R[Y] | R[Z]; break; // OR
case 0x0B: R[X] = R[Y] ^ R[Z]; break; // XOR
case 0x0C: R[X] = (R[Y] == R[Z]) ? 1 : 0; break; // EQU
case 0x0D: R[X] = (R[Y] != R[Z]) ? 1 : 0; break; // NEQU
case 0x0E: R[X] = (_signed(R[Y]) < _signed(R[Z])) ? 1 : 0; break; // LESS
case 0x0F: R[X] = (_unsigned(R[Y]) < _unsigned(R[Z])) ? 1 : 0; break; // BLW
case 0x10: R[X] = (_signed(R[Y]) > _signed(R[Z])) ? 1 : 0; break; // GRE
case 0x11: R[X] = (_unsigned(R[Y]) > _unsigned(R[Z])) ? 1 : 0; break; // ABV
case 0x12: R[X] = (_signed(R[Y]) <= _signed(R[Z])) ? 1 : 0; break; // LEQ
case 0x13: R[X] = (_unsigned(R[Y]) <= _unsigned(R[Z])) ? 1 : 0; break; // BEQ
case 0x14: R[X] = (_signed(R[Y]) >= _signed(R[Z])) ? 1 : 0; break; // GEQ
case 0x15: R[X] = (_unsigned(R[Y]) >= _unsigned(R[Z])) ? 1 : 0; break; // AEQ
case 0x16: PC = R[Y]; break; // JMP
case 0x17: R[0x1F] = PC; PC = R[Y]; break; // CALL
// same with constants
case 0x18: R[X] = R[Y] + C16S; break; // ADDE
case 0x19: R[X] = R[Y] + C16; break; // ADD
case 0x1A: R[X] = R[Y] - C16S; break;// SUBE
case 0x1B: R[X] = R[Y] - C16; break; // SUB
case 0x1C: R[X] = _unsigned(R[Y]) << C5; break; // SHL
case 0x1D: R[X] = _unsigned(R[Y]) >> C5; break; // SHR
case 0x1E: R[X] = _signed(R[Y]) >> C5; break; // SAR
case 0x1F: R[X] = R[Y] & C16; break; // AND
case 0x20: R[X] = R[Y] | C16; break; // OR
case 0x21: R[X] = R[Y] ^ C16; break; // XOR
case 0x22: R[X] = (_signed(R[Y]) == C16S) ? 1 : 0; break; // EQU
case 0x23: R[X] = (_signed(R[Y]) != C16S) ? 1 : 0; break; // NEQU
case 0x24: R[X] = (_signed(C16S) > _signed(R[Y])) ? 1 : 0; break; // GRE
case 0x25: R[X] = (_unsigned(C16) > _unsigned(R[Y])) ? 1 : 0; break; // ABV
case 0x26: R[X] = (_signed(C16S) < _signed(R[Y])) ? 1 : 0; break; // LESS
case 0x27: R[X] = (_unsigned(C16) < _unsigned(R[Y])) ? 1 : 0; break; // BLW
case 0x28: R[X] = (_signed(C16S) >= _signed(R[Y])) ? 1 : 0; break; // GEQ
case 0x29: R[X] = (_unsigned(C16) >= _unsigned(R[Y])) ? 1 : 0; break; // AEQ
case 0x2A: R[X] = (_signed(C16S) <= _signed(R[Y])) ? 1 : 0; break; // LEQ
case 0x2B: R[X] = (_unsigned(C16) <= _unsigned(R[Y])) ? 1 : 0; break; // BEQ
case 0x2C: PC += C26S; break; // JMP
case 0x2D: R[0x1F] = PC; PC += C26S; break; // CALL

case 0x2E: if (R[Y] == 0) PC += C16S; break; // JZ
case 0x2F: if (R[Y] != 0) PC += C16S; break; // JNZ
case 0x30: R[X] = I; break; // STR (correct??)
case 0x31: // LBSE
case 0x32: // LB
// correct?? (the ^ 0x03 appears strange...)
addr = &mem[((R[Y] + C16S) & 0x3FFFFF) ^ 0x03];
b = addr[0];
if (C == 0x31 && b >= 0x80) { // process sign
b-= 0x100;
}
R[X] = b;
break;
case 0x33: // LWSE
case 0x34: // LW
// correct??
addr = &mem[((R[Y] + C16S) & 0x3FFFFE) ^ 0x02];
b = (int)makeint(0, 0, addr[1], addr[0]);
if (C == 0x33 && b >= 0x8000) { // process sign
b -= 0x10000;
}
R[X] = b;
break;
case 0x35: // LDW
addr = &mem[(R[Y] + C16S) & 0x3FFFFC];
R[X] = makeint(addr[3], addr[2], addr[1], addr[0]);
break;
default: break; // NOP or unknown
}
}
}

int main() {
// TODO
}

Oopho2ei

31st August 2008, 15:04

Regarding the functionality of commands 36-3A, there are still operations missing to store values in memory, to perform systems calls and to execute native code.
I have updated commands 0x31-0x35 (improved syntax) and added the remaining commands 0x36-0x3A.

It would be interesting to know how the VM deals with arithmetic overflows (ADD, MUL)... or if they are simply not allowed in BD+.
Exceptions are handled by the interpreter but i haven't documented them yet.

Is the PC counted in bytes or words? In the below implementation it's bytes (as in your tutorial). However, this would unnecessarily restrict the range of jump commands.
Actually the PC is double word (32 bit) aligned. Every instruction is 32 bit long anyway. In my notes the PC is 4 bytes but the lower and higher bits are masked (PC & 0x3FFFFC). Have a look at what the dispatcher does:
previous instruction handler returns (has maybe changed PC)
...
command dispatcher is called:
...
PC = PC AND 0x3FFFFC; // make it dword aligned
I = mem[PC] ^ instruction_filter_value;
PC = PC + 4;
...
command_handler(I >> 0x1A);
....
command handler returns and dispatcher is called again. and so on...
Ignore the "instruction_filter_value". It's set by command ??? and is zero in my case.

Below is a simple, experimental C++ implementaion of the BD+ VM based on your specs (compiles, but not tested). Targets were keeping it short, maintainable and close to your notation. Hope I got all the signs right... Not sure about the commands for loading values from memory...
Great! :)
But you should probably use your own notation and keep my weird looking/confusing notations as comments. :)

I will record a trace of the first few commands executed by the virtual machine. Like 0000:1000 b0 00 00 40 -> PC = 0x1044
0000:1044 bc 00 ff fc -> no change
0000:1048 ...
We will use this for debugging. The 00000.svm is here: http://uploaded.to/?id=jqstc8

Edit: please use "cmd" for command and not "C" because i use "C" already otherwise. thanks.

Oopho2ei

31st August 2008, 17:03

Here is a short trace of the beginning of the VM execution. It should be enough to get the machine started. PC = 0x00001000
WD = 0x7FFFFFFF

0000:1000 B0000040 WD=0x7FFFFFFD; PC=0x00001044;
0000:1044 BC00FFFC WD=0x7FFFFFFB;
0000:1048 BC01FFFC WD=0x7FFFFFF9;
0000:104C BC02FFFC WD=0x7FFFFFF7;
0000:1050 BC03FFFC WD=0x7FFFFFF5;
0000:1054 BC04FFFC WD=0x7FFFFFF3;
0000:1058 BC05FFFC WD=0x7FFFFFF1;
0000:105C BC06FFFC WD=0x7FFFFFEF;
0000:1060 BC07FFFC WD=0x7FFFFFED;
0000:1064 BC08FFFC WD=0x7FFFFFEB;
0000:1068 BC09FFFC WD=0x7FFFFFE9;
0000:106C BC0AFFFC WD=0x7FFFFFE7;
0000:1070 BC0BFFFC WD=0x7FFFFFE5;
0000:1074 BC0CFFFC WD=0x7FFFFFE3;
0000:1078 BC0DFFFC WD=0x7FFFFFE1;
0000:107C BC0EFFFC WD=0x7FFFFFDF;
0000:1080 BC0FFFFC WD=0x7FFFFFDD;
0000:1084 BC10FFFC WD=0x7FFFFFDB;
0000:1088 BC11FFFC WD=0x7FFFFFD9;
0000:108C BC12FFFC WD=0x7FFFFFD7;
0000:1090 BC13FFFC WD=0x7FFFFFD5;
0000:1094 BC14FFFC WD=0x7FFFFFD3;
0000:1098 BC15FFFC WD=0x7FFFFFD1;
0000:109C BC16FFFC WD=0x7FFFFFCF;
0000:10A0 BC17FFFC WD=0x7FFFFFCD;
0000:10A4 BC18FFFC WD=0x7FFFFFCB;
0000:10A8 BC19FFFC WD=0x7FFFFFC9;
0000:10AC BC1AFFFC WD=0x7FFFFFC7;
0000:10B0 BC1BFFFC WD=0x7FFFFFC5;
0000:10B4 BC1CFFFC WD=0x7FFFFFC3;
0000:10B8 BC1DFFFC WD=0x7FFFFFC1;
0000:10BC BC1EFFFC WD=0x7FFFFFBF;
0000:10C0 BC1FFFFC WD=0x7FFFFFBD;
0000:10C4 C3A00000 WD=0x7FFFFFBC; R[0x1F]=0x00000000;
0000:10C8 67BD115C WD=0x7FFFFFBB; R[0x1D]=0x0000115C;
0000:10CC C3800000 WD=0x7FFFFFBA; R[0x1C]=0x00000000;
0000:10D0 679C1000 WD=0x7FFFFFB9; R[0x1C]=0x00001000;
0000:10D4 D7BD0000 WD=0x7FFFFFB7; R[0x1D]=0x6FBD001C;
0000:10D8 E3BC0000 WD=0x7FFFFFB5; mem[0x001000]=0x6FBD001C;
0000:10DC D7A10000 WD=0x7FFFFFB3; R[0x1D]=0x00000000;
0000:10E0 C3A0001F WD=0x7FFFFFB2; R[0x1D]=0x001F0000;
0000:10E4 67BDFFFC WD=0x7FFFFFB1; R[0x1D]=0x001FFFFC;
0000:10E8 B4000008 WD=0x7FFFFFAE; PC=0x000010F4; R[0x1F]=0x000010EC;
0000:10F4 D43F0000 WD=0x7FFFFFAC; R[0x01]=0x00000000;
0000:10F8 B8010018 WD=0x7FFFFFAA; PC=0x00001114;
0000:1114 6FBDFFFC WD=0x7FFFFFA9; R[0x1D]=0x001F0000;
0000:1118 C3800000 WD=0x7FFFFFA8; R[0x1C]=0x00000000;
0000:111C 679C1160 WD=0x7FFFFFA7; R[0x1C]=0x00001160;
0000:1120 C0200000 WD=0x7FFFFFA6; R[0x01]=0x00000000;
0000:1124 64211040 WD=0x7FFFFFA5; R[0x01]=0x00001040;
0000:1128 D4210000 WD=0x7FFFFFA3; R[0x01]=0x00000000;
0000:112C 6FBD0004 WD=0x7FFFFFA2; R[0x1D]=0x001EFFFC;
0000:1130 E03D0000 WD=0x7FFFFFA0; mem[0x1EFFFC]=0x00000000;
0000:1134 B4000004 WD=0x7FFFFF9D; PC=0x0000113C; R[0x1F]=0x00001138;
The watchdog (WD) is decremented by the corresponding amount of cycles after the execution of every instruction according to the table below. As long as the watchdog is greater than zero execution continues. Otherwise it breaks.
Edit: the table is:0x01,0x01,0x01,0x04,0x10,0x10,0x01,0x01, // cmd00-cmd07
0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01, // cmd07-cmd0F
0x01,0x01,0x01,0x01,0x01,0x01,0x02,0x03, // cmd10-cmd17
0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01, // cmd18-cmd1F
0x01,0x01,0x01,0x01,0x01,0x01,0x01,0x01, // cmd20-cmd27
0x01,0x01,0x01,0x01,0x02,0x03,0x02,0x02, // cmd28-cmd2F
0x01,0x04,0x04,0x04,0x04,0x02,0x04,0x04, // cmd30-cmd37
0x02,0x00,0x06,0x01,0x01,0x01,0x01,0x01 // cmd38-cmd3F

Cmd39 resets the timer back to 7FFFFFFFh. Btw the command 0x30 was wrong. I forgot a shift. It's fixed now.

ala42

31st August 2008, 18:07

It might make sense to use the DLX mnemonics, like at http://www.csee.umbc.edu/courses/undergraduate/411/spring96/dlx.html and http://www.ee.byu.edu/class/ee428/labs/DLXinst.html.

Oopho2ei

31st August 2008, 20:08

evdberg

31st August 2008, 21:14

X = ((I >> 0x13) & 0x7C) >> 2
Y = ((I >> 0x0E) & 0x7C) >> 2
Z = ((I >> 0x09) & 0x7C) >> 2

Why not simply:
X = (I >> 21) & 31
Y = (I >> 16) & 31
Z = (I >> 11) & 31

Just returned from holiday and very pleased to see all your work! I assume you all got it from the patent description?

Oopho2ei

31st August 2008, 21:53

Why not simply:
X = (I >> 21) & 31
Y = (I >> 16) & 31
Z = (I >> 11) & 31
It's equivalent. The >> 2 does the conversion from a byte address to the corresponding address in a dword (32 bit) aligned memory structure. In this case our Register array uint32 R[32]. This is really confusing sometimes. I hope the implementation does somebody else. :)
Just returned from holiday and very pleased to see all your work! I assume you all got it from the patent description?
Most of it is from compromised players. I won't give any details about which and how. Don't even try. :)

evdberg

31st August 2008, 23:00

Which is the TRAP instruction? According to the patent "one exemplary embodiment" (BD+?) would use the DLX "trap" instruction to provide access to external procedure calls (see item 127).

Oopho2ei

1st September 2008, 07:55

Which is the TRAP instruction? According to the patent "one exemplary embodiment" (BD+?) would use the DLX "trap" instruction to provide access to external procedure calls (see item 127).
I don't know but cmd39 (called "INSTF" here) looks weird. It is also executed quite often although the instruction filter value is set to 0 all the time.
Anyway I am pretty sure i observed trap_xorblock(...) doing the decryption of section 2 in file 00001.vsm so i would only need to trace it back to an instruction.

A general note:
I am trying to make a top-down approach (to get an overview) adding more details with every sweep. The exception handling for the instructions is still missing too as you might have noticed. Furthermore the implementation needs to be close to 'perfect' because every mistake which distinguishes our interpreter from a licensed one can be targeted by future bd+ movie releases. :rolleyes:

schluppo

1st September 2008, 20:31

nice work... i was thinking that maybe some kind of disasm for .svm-files would be appreciated.

so, which parts of the .svm-files are encrypted and is the decryption done by feeding the few unencrypted setup commands in the beginning of the file to the vm or is it always (i.e. for the complete file(s)) happening in the standard way that you mentioned?

----
(by standard way, i mean: )

Instruction I: C R[X], R[Y], R[Z]
C = I >> 0x1A
X = ((I >> 0x13) & 0x7C) >> 2
Y = ((I >> 0x0E) & 0x7C) >> 2
Z = ((I >> 0x09) & 0x7C) >> 2
----

Oopho2ei

1st September 2008, 21:07

nice work... i was thinking that maybe some kind of disasm for .svm-files would be appreciated.
Yes this would certainly be helpful.

so, which parts of the .svm-files are encrypted and is the decryption done by feeding the few unencrypted setup commands in the beginning of the file to the vm or is it always (i.e. for the complete file(s)) happening in the standard way that you mentioned?
It currently looks like the machine decrypts itself (at least part of it's data) with the help of traps. Furthermore there is a "instruction filter" which is a 32 bit value that is xored with the current 'obfuscated' instruction to decrypt it. This value can be updated dynamically and was introduced to make static code analysis impossible. But if your program can run 'on top' of our interpreter/emulator i don't see any problems. By running 'on top' i mean your disassembler can access usefull information of the emulator at any time and maybe store it for later use.

----
(by standard way, i mean: )

Instruction I: C R[X], R[Y], R[Z]
C = I >> 0x1A
X = ((I >> 0x13) & 0x7C) >> 2
Y = ((I >> 0x0E) & 0x7C) >> 2
Z = ((I >> 0x09) & 0x7C) >> 2
----
Please use the much clearer description of posting 27 of this thread (link: http://forum.doom9.org/showthread.php?p=1177737#post1177737)

Oopho2ei

1st September 2008, 21:19

Which is the TRAP instruction? According to the patent "one exemplary embodiment" (BD+?) would use the DLX "trap" instruction to provide access to external procedure calls (see item 127).
Actually cmd39 is the trap instruction. I was simply fooled by the save and restore of the VM state (which includes the current instruction filter). :stupid:

The trap id seems to be the immediate value. I found a jump/vector table in the vm code of my sample blue ray disk calling cmd39 with the following immediate values:
000010 = void TRAP_Finished();
000020
000110 = UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);
000120 = UINT32 TRAP_PrivateKey(UINT32 keyID, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord);
000130 = UINT32 TRAP_Random(UINT8 *dst, UINT32 len);
000140 = UINT32 TRAP_Sha(UINT8 *dst, UINT8 *src, UINT32 len, UINT32 op);
000210 = UINT32 TRAP_AddWithCarry(UINT32 *dst, UINT32 *src, UINT32 len);
000220 = UINT32 TRAP_MultiplyWithRipple(UINT32 *dst, UINT32 *src, UINT32 len, UINT32 multiplicand);
000230 = UINT32 TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len);
000310 = UINT32 TRAP_Memmove(UINT8 *dst, UINT8 *src, UINT32 len);
000320 = UINT32 TRAP_MemSearch(UINT8 *Region, UINT32 RegionLen, UINT8 *SearchData, UINT32 SearchDataLen, UINT32 *Dst);
000330 = UINT32 TRAP_Memset(UINT8 *dst, UINT8 fillvalue, UINT32 len);
000410 = UINT32 TRAP-SlotAttach(UINT32 slot, UINT32 codeLen);
000420 = UINT32 TRAP-SlotRead(UINT8 *dst, UINT32 slot);
000430 = UINT32 TRAP_SlotWrite(UINT8 *newContents);
000510 = UINT32 TRAP_DeviceAccess(UINT32 dev, UINT32 opID, UINT8 *buf);
000520 = UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);
000530 = UINT32 TRAP_DiscoveryRAM(UINT32 unknown, UINT32 dst, UINT32 len);
000540 = UINT32 TRAP_MediaReadFile(UINT8 *FileName, UINT32 SectionNumber, UINT32 Unknown, UINT32 *len, UINT8 *dst);
000550 = UINT32 TRAP_MediaSHAFileHash(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst);
000560 = UINT32 TRAP_RunNative(UINT8 *signature, UINT32 sigLen, UINT8 *code, UINT32 codeLen);
000570
008010 = UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
008020
008030
For example "E4000230" would be the call for trap_xorblock. I don't know about the others yet. There may be more...

This is a small example of some instructions between two trap calls (trap 0x110 and trap 0x230) to show how parameters are passed:
0001:ABEC E4000110 TRAP #110h // call trap 0x110
0001:ABF0 581F0000 JR r31 // return to the previous address
0001:B374 04800800 ADD r4,r0,r1
0001:B378 63BD0014 ADDIE r29,#14h // r29 = stack pointer?
0001:B37C BC010038 BNEZ r1,#0001B3B8
0001:B380 63BDFFF4 ADDIE r29,#FFFFFFF4h
0001:B384 E15D0000 SDW r10,r29,#0h // first parameter: *dst
0001:B388 E17D0004 SDW r11,r29,#4h // second parameter: *src
0001:B38C 74270002 SRLI r1,r7,#2h
0001:B390 E03D0008 SDW r1,r29,#8h // third parameter: len
0001:B394 B7FFF8EC JAL #0001AC84 // jump into trap vector table
0001:AC84 E4000230 TRAP #230h // execute trap 0x230 ( TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len) )
I wish i already had schluppo's disassembler :)

evdberg

1st September 2008, 23:05

Should JAL #PC+3FFF8EC not be JAL #PC-714 ? Considering the immediate value is 26 bits, and offsets are usually signed? And considering the address space is only 4MB (22 bits PC).

Oopho2ei

1st September 2008, 23:30

Should JAL #PC+3FFF8EC not be JAL #PC-714 ? Considering the immediate value is 26 bits, and offsets are usually signed? And considering the address space is only 4MB (22 bits PC).
You are right (again :) ). I just tried to uniformly represent the immediate part as unsigned value. You would get the same result after the addition and & 0x3FFFFC. Before any memory access you need to calculate "address = address & 0x3FFFFC" anyway. (Except if you want to access 16 bit or 8 bit values then of course you wouldn't mask the least significant 2 bits). But it is much easier to read so i corrected it. I didn't record the program counter otherwise i would have just replaced #PC-714h with the result.

Disabled

2nd September 2008, 17:48

Very nice work, especially to Oopho2ei!
Till the 12th post in this thread I never thought this would become anything at all. (And I still don't know how much is done after the bdvm is implemented.)
First I thought about trying to continue the bdvm, but then my little C knowledge kept me from continuing. (And I never was sure if anyone else is working on it.)
So I took an easier task and tried to start with the disassembler.
I started with the source from the bdvm but now very few is left. Here it goes:
#include <iostream>
#include <sstream>
#include <string>
#include <fstream>
using namespace std;

typedef unsigned char byte; // for less typing
typedef unsigned int uint; // dito

class BDDasm {
public:
BDDasm();
string dasmCmd(uint);
void dasmFile(string, uint, uint);
uint makeint(byte, byte, byte, byte);
private:
uint pc;

};
// end header

int _signed(uint x) { return (int)x; }
int _signed(int x) { return x; }
int _unsigned(uint x) { return x; }
int _unsigned(int x) { return (uint)x; }

BDDasm::BDDasm(){
pc = 0;
}

uint BDDasm::makeint(byte b3, byte b2, byte b1, byte b0) {
return ((int)b0) << 24 | ((int)b1) << 16 | ((int)b2) << 8 | ((int)b3);
}

void BDDasm::dasmFile(string fname, uint pep, uint len){
ifstream file;
file.open(fname.c_str());
file.seekg(pep);
pc = pep-6*4;
uint I; //Instruction
while(pc <= pep+len){
pc +=4;
I = makeint((byte)file.get(),(byte)file.get(),(byte)file.get(),(byte)file.get());
cout << dasmCmd(I);
}
file.close();
}

string BDDasm::dasmCmd(uint I) {

std::ostringstream out;
char buffer [100];

// instruction fetch
uint C = I >> 0x1A; // command
uint Rd = ((I >> 0x13) & 0x7C) >> 2; // operand X
uint Rs1 = ((I >> 0x0E) & 0x7C) >> 2; // operand Y
uint Rs2 = ((I >> 0x09) & 0x7C) >> 2; // operand Z
uint C16 = I & 0xFFFF; // 16 bit constant
int C16S = _signed(_unsigned(I) << 16) >> 16; // same signed
uint C5 = I & 0x1F; // 5 bit constant
uint C26 = I & 0x3FFFFFF; // 26 bit constant
int C26S = _signed(_unsigned(I) << 6) >> 6; // same signed

sprintf (buffer, "%6X ", pc-4);
out << buffer;
sprintf (buffer, "%8X ", I);
out << buffer;
// instructions
string ops[0x3C] = {"NOP", "ADD", "SUB", "MUL", "IDIV", "DIVU",
"SLL", "SRL", "SRA", "AND", "OR", "XOR", "SEQ", "SNE", "SLT",
"SBT", "SGT", "SAT", "SLE", "SBE", "SGE", "SAE", "JR", "JALR",
"ADDIE", "ADDI", "SUBIE", "SUBI", "SLLI", "SRLI", "SRAI",
"ANDI", "ORI", "XORI", "SEQI", "SNEI", "SLI", "SBI", "SGI",
"SAI", "SLEI", "SBEI", "SGEI", "SAEI", "J", "JAL", "BEQZ",
"BNEZ", "LHI", "LBE", "LB", "LWE", "LW", "LDW", "SB", "SW",
"SDW", "TRAP", "SWDR", "unknown"};

if(C>=0x00 && C<=0x3A){
out << ops[C];
}
else {
out << ops[0x3b];
}

if(C>=0x01 && C<=0x15){
out << " R" << Rd << ", R" << Rs1 << ", R" << Rs2;
}
else if(C==0x16||C==0x17){
out << " R" << Rs1;
}
else if(C>=0x18 && C<=0x3A && C!=0x2c && C!=0x2d && C!=0x39 && C!=0x2e && C!=0x2f){
sprintf(buffer, ", #%Xh", C16);
out << " R" << Rd << ", R" << Rs1 << buffer;
}
else if(C==0x2c || C==0x2d){
sprintf(buffer, " #%Xh", (pc+(C16S& 0x3FFFFC))& 0x3FFFFC);
out << buffer;
}
else if(C==0x2e || C==0x2f){
sprintf(buffer, ", #%Xh", (pc+(C16S& 0x3FFFFC))& 0x3FFFFC);
out << " R" << Rs1 << buffer;
}
else if(C==0x39){
sprintf(buffer, " #%Xh", C26);
out << buffer;
}
out << endl;
return out.str();
}

int main() {
BDDasm a;

a.dasmFile("00000.svm",0x1000,0xF000);

}

Now you can laugh at my coding style. Whats left to be done is some improvements to the dasm and of course a file reader and much more.
For a start, here is the output of the hardcoded commands:
1000 E4000110 TRAP #110h
1004 42445356 SGT R18, R4, R10
1008 581F0000 JR R31
100C 4800800 ADD R4, R0, R1
1010 63BD0014 ADDIE R29, R29, #14h
1014 BC010038 BNEZ R0, R1, #38h
1018 63BDFFF4 ADDIE R29, R29, #FFF4h
101C E15D0000 SDW R10, R29, #0h
1020 E17D0004 SDW R11, R29, #4h
1024 74270002 SRLI R1, R7, #2h
1028 E03D0008 SDW R1, R29, #8h
102C B7FFF8EC JAL, #918h
1030 E4000230 TRAP #230h

Looks quite familliar to what Oopho decoded, except for the JR and some constants.

*edit* Fixed JR handling...

*edit2* Added trivial stream handling... and watch out if someone ever tries to use the experimental BDVM, b2 is used twice in the makeint function...

*edit3* Added Offset to adress display and executed pc+r1 for jumps

*edit4* Final update with file reader and correct jumps (I hope). There is an offset for the pc of 6dwords to align with the trace in post #35, I guess thats some kind of header or something... I will post the dasm of the first 0xF000 bytes in post #40

chavonbravo

2nd September 2008, 18:08

Wow, gone a few days and all this progress.
This being a source code solution for the future, it will probably turn into a cat and mouse game with future bd+ movies, where flaws will be found and attacked, and will have to be patched up.
I beleive Peter meant before about once you're past a certain point, bd+ only helps, is that at the beginning of the vm code are all the checks for a valid player, and non-modified firmware. After you successfully emulate a valid player and pass the values needed for the disc to believe it's safe, you use the bd+ code to decypher the video stream.
This is probably why anydvd comes out with updates whenever a new bd+ disc can't be played, same cat and mouse game.

Oopho2ei

2nd September 2008, 19:35

40 B7FFF8EC JAL, #PC+FFFFF8ECh
I regret i didn't record the program counter (pc). Maybe i will add it later. In your case the PC = 40 so you could just evaluate the expression "PC+FFFFF8ECh" :)
Looks quite familliar to what Oopho decoded, except for the JR and some constants.
Please also look at some DLX disassembly examples. Like this one in section 2.2: http://heather.cs.ucdavis.edu/~matloff/DLX/Report.html and others.

Thank you for your effort. A disassembler will be helpful to study the traps.

This being a source code solution for the future, it will probably turn into a cat and mouse game with future bd+ movies, where flaws will be found and attacked, and will have to be patched up.
This is likely to happen but at least people will know how BD+ works. All discs which can be decrypted up to that point can also be decrypted in the future. I can't prevent anyone from writing and selling proprietary software based on the information from this thread. But keep in mind that if the interpreter is open source everyone can make modifications maybe based on research of other licensed players. Making it more difficult to target all those different version. It gets more people involved and also improves the knowledge of the community. Finally if your open source implementation has been targeted and is not working anymore for newer discs you would "only" need to record and compare the instruction trace (and maybe register values) of both your implementation and a working licensed player to find the exploit.

Anyway:
To guide your implementation efforts i dumped the register file and the machine memory directly before the very first trap instruction (trap 110h in this case) is executed.

register file (32 register each 32 bit): http://uploaded.to/?id=8kkmfn
machine memory (4MB): http://uploaded.to/?id=nqpu3z
execute this from offset 0x1000: http://uploaded.to/?id=jqstc8

Keep in mind that the players do a lot of big endian <-> little endian conversion (improves vm speed probably) . When you look at offset 0x55D0 of the vm_mem_dump.000 you will know what i mean.

Btw i have added some more comments on the few instruction between two trap instruction calls in posting #34. Some of you have probably already guessed where the parameters are stored and in which order. :rolleyes:
The parameters are written to the address [r29] in the order given by the function declaration. Maybe think of r29 as ESP (stack pointer) and every SDW (store double word) here would be a push. (This is not true in general of course but for calling a trap function. I haven't verified that for other traps yet.)

Disabled

2nd September 2008, 21:06

In your case the PC = 40 so you could just evaluate the expression "PC+FFFFF8ECh" :)
I added that...

Please also look at some DLX disassembly examples...
Will do...

I tried to dissasemble your dump and something was fishy:
1000 0 NOP
1004 0 NOP
1008 0 NOP
100C 0 NOP
1010 0 NOP
1014 0 NOP
1018 B0000040 J #1058h
101C E03D0008 SDW R1, R29, #8h
1020 E3FD0004 SDW R31, R29, #4h
1024 D4200000 LDW R1, R0, #0h
1028 E01D0000 SDW R0, R29, #0h
102C D7E01154 LDW R31, R0, #1154h
1030 E3E01150 SDW R31, R0, #1150h
1034 B4019B0C JAL #3FAB40h
1038 D7E01158 LDW R31, R0, #1158h
103C E3E01150 SDW R31, R0, #1150h
1040 E0200004 SDW R1, R0, #4h
1044 D7FD0004 LDW R31, R29, #4h
1048 D43D0008 LDW R1, R29, #8h
104C 67BD001C ADDI R29, R29, #1Ch
1050 E4000010 TRAP #10h
1054 B3FFFFFC J #1050h
1058 0 NOP
105C BC00FFFC BNEZ R0, #1058h
1060 BC01FFFC BNEZ R1, #105Ch
1064 BC02FFFC BNEZ R2, #1060h
1068 BC03FFFC BNEZ R3, #1064h
106C BC04FFFC BNEZ R4, #1068h
1070 BC05FFFC BNEZ R5, #106Ch
1074 BC06FFFC BNEZ R6, #1070h
1078 BC07FFFC BNEZ R7, #1074h
107C BC08FFFC BNEZ R8, #1078h
1080 BC09FFFC BNEZ R9, #107Ch
1084 BC0AFFFC BNEZ R10, #1080h
1088 BC0BFFFC BNEZ R11, #1084h
108C BC0CFFFC BNEZ R12, #1088h
1090 BC0DFFFC BNEZ R13, #108Ch
1094 BC0EFFFC BNEZ R14, #1090h
1098 BC0FFFFC BNEZ R15, #1094h
109C BC10FFFC BNEZ R16, #1098h
10A0 BC11FFFC BNEZ R17, #109Ch
10A4 BC12FFFC BNEZ R18, #10A0h
10A8 BC13FFFC BNEZ R19, #10A4h
10AC BC14FFFC BNEZ R20, #10A8h
10B0 BC15FFFC BNEZ R21, #10ACh
10B4 BC16FFFC BNEZ R22, #10B0h
10B8 BC17FFFC BNEZ R23, #10B4h
10BC BC18FFFC BNEZ R24, #10B8h
10C0 BC19FFFC BNEZ R25, #10BCh
10C4 BC1AFFFC BNEZ R26, #10C0h
10C8 BC1BFFFC BNEZ R27, #10C4h
10CC BC1CFFFC BNEZ R28, #10C8h
10D0 BC1DFFFC BNEZ R29, #10CCh
10D4 BC1EFFFC BNEZ R30, #10D0h
10D8 BC1FFFFC BNEZ R31, #10D4h
10DC C3A00000 LHI R29, R0, #0h
10E0 67BD115C ADDI R29, R29, #115Ch
10E4 C3800000 LHI R28, R0, #0h
10E8 679C1000 ADDI R28, R28, #1000h
10EC D7BD0000 LDW R29, R29, #0h
10F0 E3BC0000 SDW R29, R28, #0h
10F4 D7A10000 LDW R29, R1, #0h
10F8 C3A0001F LHI R29, R0, #1Fh
10FC 67BDFFFC ADDI R29, R29, #FFFCh
1100 B4000008 JAL #1108h
1104 0 NOP
...

Are you sure its really PC += value with jumps? I don't know much asm, but I always thought they were just PC = value or something with an absolute offset like the program entry point or a memory offset... But even then that code doesn't make much sense...

Another question: In the .svm files, are there just instructions or is there data that has not length%4==0? Else I could try to implement a quick jump label generation. If there is data whose length is not dividable by 4 I'm sure I wouldn't be able to do this, as I would have to really analyze the code and decide where to start again with the dasm. Else a quick jump-search-prepass would do.

*edit* he is right and here is the (hopefully) correct dasm of the first 0xF000 bytes:
dasm.zip - 0.14MB (http://www.zshare.net/download/180874810c41c8b9/)

superman500

2nd September 2008, 21:47

Great job so far!

Oopho2ei

2nd September 2008, 21:58

105C BC00FFFC BNEZ R0, #1058h
Please look at the pseudocode in posting #24. The program counter is already incremented during the fetch stage so when the execution of the current instruction (in this case 'BNEZ') starts the PC has already been incremented by 4. Like in the old (bad syntax) that would be "BNEZ R0,#(PC+4)-4" So this instruction actually checks if R0 is zero. If it is everything is fine and the next register is checked. If it is not zero then the machine enters a infinite loop in other words halts. The instruction actually is:
105C BC00FFFC BNEZ R0, #105Ch
The same applies to "1018 B0000040 J #1058h" you are always jumping one instruction too short. :)
If i see more mistakes i will edit this posting. Sorry that the information is scattered all over this thread (and sometimes more confusing than helpful :) ). Maybe somebody could make a summery.

Another question: In the .svm files, are there just instructions or is there data that has not length%4==0?
When you look at the 00000.svm file from my last posting you will find strings (copyright foo bar...) so yes there is data. I don't know if there is any data which is not dword aligned but remember that every instruction is dword aligned (instruction address = address & 0x3FFFFC. you know). Hope that helps a little.

In posting #25 is a instruction trace starting from 0x1000 (executing that 00000.svm file). You might want to use that too.

Btw i have now added the instruction offsets for the trace between two trap calls (posting #34).

Disabled

2nd September 2008, 22:11

If it is not zero then the machine enters a infinite loop in other words halts.

And I just thought noone would programm like that... But then everything is clear.

... don't know if there is any data which is not dword aligned but remember that every instruction is word aligned ... Hope that helps a little.

dword would make it quite simple, with word alignement I would have to trace the program flow and ... well I think its pretty hard to make a jump analysis like "BNEZ r1, 2" with r1==0 then I don't know where to continue with the dasm. But that is a problem that occurs anyway when I want to dasm the whole file and I wonder how that would be solved in a professional dasm.

In posting #25 is a instruction trace starting from 0x1000 (executing that 00000.svm file). You might want to use that too.
I will do that! Nice suggestion.

btw: I got a file reader running. I will update the code once I'm a little more sure about its correctness.

Oopho2ei

2nd September 2008, 22:23

Before you invest a lot of effort in static code analysis please keep that in mind:
Instruction Filter. The Instruction Filter provides a mechanism to protect content code from static disassembly, or to enable device-specific behavior. Prior to executing an instruction, the VM first computes the XOR of the instruction with the current 32-bit Instruction Filter value. Content code may set the value of the Instruction Filter dynamically via the INSTF instruction. The Instruction filter potentially enhances a content creator’s ability to obfuscate code by forcing adversaries to trace the full code execution path rather than disassembling code in a single pass. Instruction filter values may be computed via cryptographic operations, or from device-specific operations, which may require the adversary to correctly emulate this behavior in order to disassemble code.
The instruction filter is currently zero (couldn't observe any other value) but it is unlikely it will stay that way on future releases. So you have to work together with the guys who write the emulator/interpreter.

Disabled

2nd September 2008, 22:33

Before you invest a lot of effort in static code analysis please keep that in mind:

The instruction filter is currently zero (couldn't observe any other value) but it is unlikely it will stay that way on future releases. So you have to work together with the guys who write the emulator/interpreter.

Ok, then we really can't create a disassembler that way...
perhaps I'll look into the VM the other day as it didn't look too complicated. At least I got started with cpp again and some of my code might proove usefull in the future - at least for me.
*edit* I'll wrap a few things together and release what I have till now in a few minutes...done

Oopho2ei

2nd September 2008, 23:07

"bmnot" has already started with the emulator/interpreter (see posting #23). (I don't know how far he has come.) Now instead of reading the instruction from the file you would get a sequence of (address, instruction) pairs from the emulator. You would take the instruction and place it at the given address in your machine memory. (Same for register and memory accesses.) The memory would slowly be filled with (plain) instructions and data you can analyze.
Anyway one doesn't really have to study every program run by every disc on the virtual machine. Currently it helps to see the dataflow in and out of the trap calls. But beyond that it is probably only useful to analyze exploits used to disable successful emulation. (which might happen a lot in the future if this succeeds).
What is currently needed most is a working emulator. In posting #39 i provided a memory and register file snapshot prior the execution of the first trap command (#110). If anyone makes it that far and the register/memory state of the emulator matches the given snapshot please leave a note here.

Disabled

2nd September 2008, 23:30

Of course I know about his attempts and I first started with improving it, but as I didn't write c++ code for a long time I started with something that seemed easier to me.
At the end you just can't really code a dasm for these files because as you said its a quite static code analysis and they can make the code non-static. They could even jump twice to a single dword with a different instruction filter and let it do different things. And if the code decrypts itself, dasm is also useless.

So a VM is the only way to go. Of course it can have a debugger that disassembles the current ops, but a static dasm is impossible imo.

*edit* one thing about the traps... we would have to implement them in another way than the vm do we? Or better said: The VM has to know what each trap exactly does and provide (c++)code accordingly does it?

Oopho2ei

2nd September 2008, 23:54

*edit* one thing about the traps... we would have to implement them in another way than the vm do we? Or better said: The VM has to know what each trap exactly does and provide (c++)code accordingly does it?
Yes, the traps are implemented in the host system. They provide the data interface between player (host system) and the program running on the virtual machine (guest system). Please refer to http://securityevaluators.com/pdf/spdc_aacs_2005.pdf section "The SPDC Interface Specification" for details. The traps are described more closely here: http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070033419%22.PGNR.&OS=DN/20070033419&RS=DN/20070033419 (links taken from posting #14 from "chavonbravo")
But before that i would have to figure out the mapping between the trap number and the Trap name assuming they are all listed in that patent above.

Disabled

3rd September 2008, 01:42

And here I present: An improved draft experimental alpha BDVM!

I can run your dump and the machine shows the same trace as in post #25. That doesn't mean much, as there are no conditional jumps (after I calculate something) in that trace and on the first conditional jump after a calculation it goes in an infinitive loop.
But the basics work - now I (someone) has to debug it.
I used bmnots work as a great start. I hope our developement didn't collide here.
I tried to convert some of his syntax to the new one from post #27 and tried to add the last cmds. That was before I did the dasm, so it added a lot of clutter, as I did some things I didn't really understand. I will investigate into that later.

Now the dasm is back, to tell us what happens. So here is the trace of the file:

1000 0 NOP
1004 0 NOP
1008 0 NOP
100C 0 NOP
1010 0 NOP
1014 0 NOP
1018 B0000040 J #105Ch
105C BC00FFFC BNEZ R0, #105Ch
1060 BC01FFFC BNEZ R1, #1060h
1064 BC02FFFC BNEZ R2, #1064h
1068 BC03FFFC BNEZ R3, #1068h
106C BC04FFFC BNEZ R4, #106Ch
1070 BC05FFFC BNEZ R5, #1070h
1074 BC06FFFC BNEZ R6, #1074h
1078 BC07FFFC BNEZ R7, #1078h
107C BC08FFFC BNEZ R8, #107Ch
1080 BC09FFFC BNEZ R9, #1080h
1084 BC0AFFFC BNEZ R10, #1084h
1088 BC0BFFFC BNEZ R11, #1088h
108C BC0CFFFC BNEZ R12, #108Ch
1090 BC0DFFFC BNEZ R13, #1090h
1094 BC0EFFFC BNEZ R14, #1094h
1098 BC0FFFFC BNEZ R15, #1098h
109C BC10FFFC BNEZ R16, #109Ch
10A0 BC11FFFC BNEZ R17, #10A0h
10A4 BC12FFFC BNEZ R18, #10A4h
10A8 BC13FFFC BNEZ R19, #10A8h
10AC BC14FFFC BNEZ R20, #10ACh
10B0 BC15FFFC BNEZ R21, #10B0h
10B4 BC16FFFC BNEZ R22, #10B4h
10B8 BC17FFFC BNEZ R23, #10B8h
10BC BC18FFFC BNEZ R24, #10BCh
10C0 BC19FFFC BNEZ R25, #10C0h
10C4 BC1AFFFC BNEZ R26, #10C4h
10C8 BC1BFFFC BNEZ R27, #10C8h
10CC BC1CFFFC BNEZ R28, #10CCh
10D0 BC1DFFFC BNEZ R29, #10D0h
10D4 BC1EFFFC BNEZ R30, #10D4h
10D8 BC1FFFFC BNEZ R31, #10D8h
10DC C3A00000 LHI R29, R0, #0h
10E0 67BD115C ADDI R29, R29, #115Ch
10E4 C3800000 LHI R28, R0, #0h
10E8 679C1000 ADDI R28, R28, #1000h
10EC D7BD0000 LDW R29, R29, #0h
10F0 E3BC0000 SDW R29, R28, #0h
10F4 D7A10000 LDW R29, R1, #0h
10F8 C3A0001F LHI R29, R0, #1Fh
10FC 67BDFFFC ADDI R29, R29, #FFFCh
1100 B4000008 JAL #110Ch
110C D43F0000 LDW R1, R31, #0h
1110 B8010018 BEQZ R1, #112Ch
112C 6FBDFFFC SUBI R29, R29, #FFFCh
1130 C3800000 LHI R28, R0, #0h
1134 679C1160 ADDI R28, R28, #1160h
1138 C0200000 LHI R1, R0, #0h
113C 64211040 ADDI R1, R1, #1040h
1140 D4210000 LDW R1, R1, #0h
1144 6FBD0004 SUBI R29, R29, #4h
1148 E03D0000 SDW R1, R29, #0h
114C B4000004 JAL #1154h
1154 D43F0000 LDW R1, R31, #0h
1158 B8010004 BEQZ R1, #1160h
1160 67BD0004 ADDI R29, R29, #4h
1164 B3FFFEB0 J #1018h
1018 B0000040 J #105Ch
105C BC00FFFC BNEZ R0, #105Ch
1060 BC01FFFC BNEZ R1, #1060h
1064 BC02FFFC BNEZ R2, #1064h
1068 BC03FFFC BNEZ R3, #1068h
106C BC04FFFC BNEZ R4, #106Ch
1070 BC05FFFC BNEZ R5, #1070h
1074 BC06FFFC BNEZ R6, #1074h
1078 BC07FFFC BNEZ R7, #1078h
107C BC08FFFC BNEZ R8, #107Ch
1080 BC09FFFC BNEZ R9, #1080h
1084 BC0AFFFC BNEZ R10, #1084h
1088 BC0BFFFC BNEZ R11, #1088h
108C BC0CFFFC BNEZ R12, #108Ch
1090 BC0DFFFC BNEZ R13, #1090h
1094 BC0EFFFC BNEZ R14, #1094h
1098 BC0FFFFC BNEZ R15, #1098h
109C BC10FFFC BNEZ R16, #109Ch
10A0 BC11FFFC BNEZ R17, #10A0h
10A4 BC12FFFC BNEZ R18, #10A4h
10A8 BC13FFFC BNEZ R19, #10A8h
10AC BC14FFFC BNEZ R20, #10ACh
10B0 BC15FFFC BNEZ R21, #10B0h
10B4 BC16FFFC BNEZ R22, #10B4h
10B8 BC17FFFC BNEZ R23, #10B8h
10BC BC18FFFC BNEZ R24, #10BCh
10C0 BC19FFFC BNEZ R25, #10C0h
10C4 BC1AFFFC BNEZ R26, #10C4h
10C8 BC1BFFFC BNEZ R27, #10C8h
10CC BC1CFFFC BNEZ R28, #10CCh
10CC BC1CFFFC BNEZ R28, #10CCh
10CC BC1CFFFC BNEZ R28, #10CCh

And now the code (and someone has to laugh this time!)
*edited* code removed, as its not the latest version. I don't have the latest version for download right now, but will release it instantly, if someone asks.

And here is the trace with register dump:
out.txt (http://www.zshare.net/download/18093691a1a910e8/)

You can very easily change the .c file to act as a non interactive debugger.

Thats it for today from me.

Oopho2ei

3rd September 2008, 07:56

I can run your dump and the machine shows the same trace as in post #25. That doesn't mean much, as there are no conditional jumps (after I calculate something) in that trace and on the first conditional jump after a calculation it goes in an infinitive loop.
Ok i will post a longer trace soon. The infinite loop suggests that the emulator did something wrong :rolleyes:

Btw. you missed something i explained in posting #22: The 00000.svm file has a 18h byte long header so when i say the execution starts at 0x1000 this translates to address 0x1000+0x18=0x1018 in the file.

Thats it for today from me.
Looks good. Thank you. :)

Disabled

3rd September 2008, 11:27

Btw. you missed something i explained in posting #22: The 00000.svm file has a 18h byte long header ...

Yeah I really missed that but it didn't matter anyway, as 24 is mod 4 and there were just nops. But I corrected that.

A longer trace would be greatly apreciated. I try to evaluate the asm myself, but I don't find an error.

Disabled

3rd September 2008, 12:56

One important question (and thus I'm not editing the post above):
Is the programm loaded to memory bevor execution? IE is the mem of the VM initialized with zeros, or is one section copied to memory before execution?
That would give the trace at least a bit of sense - but doesn't change anything, it loops on the same way. I'm trying to find a bug in the sdw cmd now (as thats one I added to bmnots code).

*edit* I think with some more hacks to the code I got it to execute a little longer. I indeed loaded one code section to memory and executed from there. Now it segfaults due to a jump to 0x8000000... Here is the new trace:

1000 B0000040 J #1044h
1044 BC00FFFC BNEZ R0, #1044h
1048 BC01FFFC BNEZ R1, #1048h
104C BC02FFFC BNEZ R2, #104Ch
1050 BC03FFFC BNEZ R3, #1050h
1054 BC04FFFC BNEZ R4, #1054h
1058 BC05FFFC BNEZ R5, #1058h
105C BC06FFFC BNEZ R6, #105Ch
1060 BC07FFFC BNEZ R7, #1060h
1064 BC08FFFC BNEZ R8, #1064h
1068 BC09FFFC BNEZ R9, #1068h
106C BC0AFFFC BNEZ R10, #106Ch
1070 BC0BFFFC BNEZ R11, #1070h
1074 BC0CFFFC BNEZ R12, #1074h
1078 BC0DFFFC BNEZ R13, #1078h
107C BC0EFFFC BNEZ R14, #107Ch
1080 BC0FFFFC BNEZ R15, #1080h
1084 BC10FFFC BNEZ R16, #1084h
1088 BC11FFFC BNEZ R17, #1088h
108C BC12FFFC BNEZ R18, #108Ch
1090 BC13FFFC BNEZ R19, #1090h
1094 BC14FFFC BNEZ R20, #1094h
1098 BC15FFFC BNEZ R21, #1098h
109C BC16FFFC BNEZ R22, #109Ch
10A0 BC17FFFC BNEZ R23, #10A0h
10A4 BC18FFFC BNEZ R24, #10A4h
10A8 BC19FFFC BNEZ R25, #10A8h
10AC BC1AFFFC BNEZ R26, #10ACh
10B0 BC1BFFFC BNEZ R27, #10B0h
10B4 BC1CFFFC BNEZ R28, #10B4h
10B8 BC1DFFFC BNEZ R29, #10B8h
10BC BC1EFFFC BNEZ R30, #10BCh
10C0 BC1FFFFC BNEZ R31, #10C0h
10C4 C3A00000 LHI R29, #0h
10C8 67BD115C ADDI R29, R29, #115Ch
R29:115C,
10CC C3800000 LHI R28, #0h
R29:115C,
10D0 679C1000 ADDI R28, R28, #1000h
R28:1000, R29:115C,
10D4 D7BD0000 LDW R29, R29, #0h
R28:1000, R29:6FBD001C,
10D8 E3BC0000 SDW R29, R28, #0h
R28:1000, R29:6FBD001C,
10DC D7A10000 LDW R29, R1, #0h
R28:1000, R29:B3FFFFFC,
10E0 C3A0001F LHI R29, #1Fh
R28:1000, R29:1F0000,
10E4 67BDFFFC ADDI R29, R29, #FFFCh
R28:1000, R29:1FFFFC,
10E8 B4000008 JAL #10F4h
R28:1000, R29:1FFFFC, R31:10EC,
10F4 D43F0000 LDW R1, R31, #0h
R28:1000, R29:1FFFFC, R31:10EC,
10F8 B8010018 BEQZ R1, #1114h
R28:1000, R29:1FFFFC, R31:10EC,
1114 6FBDFFFC SUBI R29, R29, #FFFCh
R28:1000, R29:1F0000, R31:10EC,
1118 C3800000 LHI R28, #0h
R29:1F0000, R31:10EC,
111C 679C1160 ADDI R28, R28, #1160h
R28:1160, R29:1F0000, R31:10EC,
1120 C0200000 LHI R1, #0h
R28:1160, R29:1F0000, R31:10EC,
1124 64211040 ADDI R1, R1, #1040h
R1:1040, R28:1160, R29:1F0000, R31:10EC,
1128 D4210000 LDW R1, R1, #0h
R28:1160, R29:1F0000, R31:10EC,
112C 6FBD0004 SUBI R29, R29, #4h
R28:1160, R29:1EFFFC, R31:10EC,
1130 E03D0000 SDW R1, R29, #0h
R28:1160, R29:1EFFFC, R31:10EC,
1134 B4000004 JAL #113Ch
R28:1160, R29:1EFFFC, R31:1138,
113C D43F0000 LDW R1, R31, #0h
R1:1F3E8, R28:1160, R29:1EFFFC, R31:1138,
1140 B8010004 BEQZ R1, #1148h
R1:1F3E8, R28:1160, R29:1EFFFC, R31:1138,
1144 5C010000 JALR R1
R1:1F3E8, R28:1160, R29:1EFFFC, R31:1148,
1F3E8 6FBD0010 SUBI R29, R29, #10h
R1:1F3E8, R28:1160, R29:1EFFEC, R31:1148,
1F3EC E3DD000C SDW R30, R29, #Ch
R1:1F3E8, R28:1160, R29:1EFFEC, R31:1148,
1F3F0 67DD0010 ADDI R30, R29, #10h
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1148,
1F3F4 E3FEFFF8 SDW R31, R30, #FFF8h
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1148,
1F3F8 E05D0000 SDW R2, R29, #0h
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1148,
1F3FC E07D0004 SDW R3, R29, #4h
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1148,
1F400 D47E0000 LDW R3, R30, #0h
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1148,
1F404 B4001874 JAL #20C7Ch
R1:1F3E8, R28:1160, R29:1EFFEC, R30:1EFFFC, R31:1F408,
20C7C 6FBD0010 SUBI R29, R29, #10h
R1:1F3E8, R28:1160, R29:1EFFDC, R30:1EFFFC, R31:1F408,
20C80 E3DD000C SDW R30, R29, #Ch
R1:1F3E8, R28:1160, R29:1EFFDC, R30:1EFFFC, R31:1F408,
20C84 67DD0010 ADDI R30, R29, #10h
R1:1F3E8, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C88 E3FEFFF8 SDW R31, R30, #FFF8h
R1:1F3E8, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C8C E05D0000 SDW R2, R29, #0h
R1:1F3E8, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C90 C0200000 LHI R1, #0h
R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C94 64211150 ADDI R1, R1, #1150h
R1:1150, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C98 C0400002 LHI R2, #2h
R1:1150, R2:20000, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20C9C 6442047C ADDI R2, R2, #47Ch
R1:1150, R2:2047C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CA0 E0410000 SDW R2, R1, #0h
R1:1150, R2:2047C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CA4 E0410004 SDW R2, R1, #4h
R1:1150, R2:2047C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CA8 60420400 ADDIE R2, R2, #400h
R1:1150, R2:2087C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CAC E0410008 SDW R2, R1, #8h
R1:1150, R2:2087C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CB0 D45D0000 LDW R2, R29, #0h
R1:1150, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
20CB4 D7FEFFF8 LDW R31, R30, #FFF8h
R1:1150, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:8000000,
20CB8 7A0F000 ADD R29, R0, R30
R1:1150, R28:1160, R29:1EFFEC, R30:1EFFEC, R31:8000000,
20CBC D7DEFFFC LDW R30, R30, #FFFCh
R1:1150, R28:1160, R29:1EFFEC, R30:FC000000, R31:8000000,
20CC0 581F0000 JR R31

Oopho2ei

3rd September 2008, 16:49

Is the programm loaded to memory bevor execution? Of course the program is loaded into memory before execution. The program counter is the address of the instruction in the memory. I uploaded a memory snapshot in posting #39. Please have a look at it.

Now it segfaults due to a jump to 0x8000000 As stated prior in posting #24: Before you load the next instruction you adjust the program counter by calculating "PC = PC AND 0x3FFFFC;".

Hope that helps.

I will record the extended trace now. Expect a set of binary files. Each file is associated with exactly one command and contains a list of the results of this command in the order of execution. So to get the result of a particular instruction eg. "ADD Rd,Rs1,Rs2" you would have to open the file "cmd01_trace.bin" and look at address 4*i where i is the number of times this command (add) has already been executed. The 32bit value at this address would be Rd. You would need to write a program which keeps track of how many times every command has been executed. After the execution of every command your program is supposed to compare the result and abort execution once a difference has been found.

Btw. by the term "instruction" i mean "command + parameters". Just to avoid confusion ^

Disabled

3rd September 2008, 17:06

Of course the program is loaded into memory before execution.
It wasn't in the original post from bmnot and he zeroed the mem first and I didn't find what memory I should load into the mem (a 2MB segment, and zero the following 2MB, or just copy 2 segments or what?).

As stated prior in posting #24: Before you load the next instruction you adjust the program counter by calculating "PC = PC AND 0x3FFFFC;".

Hope that helps.
No it does not. It sets the pc to 8000000 and in the next step it does the 0x3ffffc. And I also changed the jump directly, it doesn't change anything, as 800000 & 0x3FFFFC = 0 and at that adress is a j 0
*edit* of course I forgot to set pc to &0x3ffffc in the next step, but as already written, that does not change anything. */edit*

I will record the extended trace now. Expect a set of binary files. Each file is associated with exactly one command and contains a list of the results of this command in the order of execution. So to get the result of a particular instruction eg. "ADD Rd,Rs1,Rs2" you would have to open the file "cmd01_trace.bin" and look at address 4*i where i is the number of times this command (add) has already been executed. The 32bit value at this address would be Rd. You would need to write a program which keeps track of how many times every command has been executed. After the execution of every command your program is supposed to compare the result and abort execution once a difference has been found.

Uhh that looks like a bit of work. How about just dumping the regs every step and comparing them at each step? And recording the PC too...

Oopho2ei

3rd September 2008, 17:20

It wasn't in the original post from bmnot and he zeroed the mem first and I didn't find what memory I should load into the mem (a 2MB segment, and zero the following 2MB, or just copy 2 segments or what?).Just load the first section (2MB) at the base (skip the header) and leave the other 2MB free.

No it does not. It sets the pc to 8000000 and in the next step it does the 0x3ffffc. And I also changed the jump directly, it doesn't change anything, as 800000 & 0x3FFFFC = 0 and at that adress is a j 0
*lol* but j #0 would be correct. You see that with all those instructions they test if your interpreter/emulator is working correctly. Just wait for the trace. We will fix those problems.

Uhh that looks like a bit of work. How about just dumping the regs every step and comparing them at each step? And recording the PC too...This is just easier to to do for me. Also you are not supposed to compare the result of every instruction yourself. Your program can do that for you. :rolleyes:

Disabled

3rd September 2008, 17:30

... leave the over 2MB+0x16 free.... right?

*lol* but j #0 would be correct.
But if I do a j #0 at adress 0, then its an infinite loop... so there has to be an error.

You see that with all those instructions they test if your interpreter/emulator is working correctly.
That took me a long time to understand. The code just didn't made sense until I realized that.

This is just easier to to do for me. Also you are not supposed to compare the result of every instruction yourself. Your program can do that for you.
The question is, who of us both has more time? :p Opening one file and comparing 33 values would be very easy to code for me, opening 0x3F files, incrementing one of 0x3F counters each step and comparing one value each step sounds a little more complex to me. (I know I can array and loop those duties too.)
On a second thought it doesn't sound too hard to do...

Oopho2ei

3rd September 2008, 17:45

A general note: Doubts have been raised about my statement that the virtual machine is "little endian" in posting #20. I have to admit that i probably made a mistake there and the virtual machine is most likely big endian. Therefor i have edited posting #20.

Please feel free to post your comments if you think i am wrong at any point. :thanks:

Disabled

3rd September 2008, 18:12

Ok, I guess I have the debug code written. Now I just need a dump to proove it...

*edit*
And another update to the VM, this time the decoded instructions count has nearly tripled. It works till the 278th instruction and that instruction is a trap. But its trap #10h and not #110 as you indicated and the registers and everything are very different. I have some ideas where the problem might be and will try to solve them. Until then, here is the new trace (http://www.zshare.net/download/181376128e9fd7c8/).

I didn't release the updated source to the vm yet, but will instantly, if someone is interested to participate.

*edit2* I think you are right with it being Big Endian. That was one of the first things I changed after I got the basic concept running, but never realized it might be something to report back - sorry.

bmnot

3rd September 2008, 21:15

Hi,

I'm glad work on this has progressed. Unfortunately, I'm pretty busy at the moment - job and changing flat - the latter effectively preventing any productive work on this topic. So currently you don't need to worry about collisions regarding coding... feel free to use the code in any way you like.
(maybe we'll need a svn some day).

Anyway, I thought a while about convenient/efficient BD+ debugging...

A graphical debugger for the VM would be interesting - with all the disassembled (or even decompiled?) instructions in a window on the left, register contents/PC/watchdog counter etc. on the right and the possibility to step through the code, set breakpoints etc.

As soon as I find some time (weekend), I'll look over the code and join the experiments with the VM.
If it turns out that this would mean significant progress, I'll implement a graphical debugger... If there are no objections, I'd do this in Java (or is anyone already working on such a debugger?).
There are two options regarding the actual VM:
- implement it in Java, too (easier to compile & debug)
- let the C++-VM provide a JNI interface
Any opinions/preferences/suggestions on this? Personally, I find Java more convenient - just dealing with signed and unsigned values is more annoying (that's why I used C++ initially).

By the way, what's actually the long term goal of the BD+-VM? I'd say, bringing it to a form in which it can be easily integrated into other programs such as KenD00's DumpHD (which is by the way Java, too).

From the patent application:

Players also provide procedure calls to enable content code to load data from media, perform network communications, determine playback environment configurations (225), access secure non-volatile storage, submit data to CODECs for output (250), and/or perform cryptographic operations.

Guess that means we'll have to find and implement some kind of "BD+ API" - with exactly the correct behaviour (so our VM cannot be distinguished from a "real" one).

Disabled

3rd September 2008, 21:37

Hi Bmnot and glad to hear your voice here.
To be honest, I shouldn't have time for this too, but I'm too lazy to do the work I should do...

A graphicall Debugger would of course be nice. On top of my code it would be pretty easy to attach an gui if I just had any experience with gui-programming. Right now you can use the main class as a non-interactive debugger(*), but the need to recompile the whole project everytime you change something to the debug instructions makes it not so easy to use, but a recompilation is needed anyways most of the time, as I change something to the main codebase way more often.
I don't mind using Java, I did far more in Java than in C++...

My long term goal with the vm is... do something until Oopho2ei tells me to add another thing... Sure if it ever is able to decrypt something it will be integrated into other tools or developed into a standalone app. Right now I have no idea what the next step will be. Fixing bugs for sure, but after that all there is left are the traps are there? (And the watchdog, but as I understand, execution just stops when the watchdog reaches 0, so what if I just ignore the watchdog? Or are there traps that can use the watchdog as input...?)
And traps are what I understand you call the BD+ API, is it?

*edit* I forgot to post the latest bdvm code (http://www.zshare.net/download/183268689ed30ac4/). I really cleaned it a lot and from my ideas this is not the worst code I've seen anymore.

(*) And I wanted to clear this a little. This is the int main():
int main() {
ExperimentalBdVM a; //create BdVM object
a.loadFile("mem"); //mem is actually a header stripped 2MB extract of 00000.svm
a.initVM(0x1000);
//a.loadMem("mem");
//a.loadRegisters("reg_file.000");
//a.debugon();

for (int i=0;i<34;i++){
a.step();
}
//a.dumpMem("dump.txt");
for (int i=0;i<250;i++){
cout << a.instruction_counter +1<<endl;
a.step();
a.dispRegisters();
}
}
The ExperimentalBdVM object has no main loop. All it can do is taking a step. Of course something setting breakpoints or something could easily be implemented. And for an actual gui debugger some of those functions would have to be adapted but still...

Oopho2ei

3rd September 2008, 22:06

pc_trace.bin: http://uploaded.to/?id=vk3u12

cmd01_trace.bin: http://uploaded.to/?id=ppqht6
cmd02_trace.bin: http://uploaded.to/?id=w66t6q
cmd19_trace.bin: http://uploaded.to/?id=dfkqk2
cmd35_trace.bin: http://uploaded.to/?id=t8scb6
cmd38_trace.bin: http://uploaded.to/?id=cp2j1y

Each "trace" extends beyond the first trap call.

It is difficult to get this so please be patient. I will post the remaining files as soon as possible.

Disabled

3rd September 2008, 22:40

Thanks a lot, that might already help. The VM is ok for 17 ADDs (that are a whooping 226 instructions) and has a bad register on the 18th with 7 instructions inbetween. I'll look into it and report back.
Btw: You recorded them in little endian, did you?

*edit* I didn't find anything usefull. There are 3 registers and one memory adress involved between the two adds that might provoke the error. And they could of course be wrong because something entirely else is wrong I don't know about...
I made a little statistic of the ops used till that point:

Op0(0):Op1(18):Op2(0):Op3(0):Op4(0):Op5(0):Op6(0):Op7(0):Op8(0):Op9(1):OpA(0):OpB(0):OpC(0):OpD(3):OpE(0):
OpF(0):Op10(0):Op11(0):Op12(0):Op13(1):Op14(0):Op15(0):Op16(4):Op17(1):Op18(9):Op19(21):Op1A(0):Op1B(8):
Op1C(4):Op1D(0):Op1E(0):Op1F(0):Op20(0):Op21(0):Op22(2):Op23(0):Op24(0):Op25(0):Op26(0):Op27(0):Op28(0):
Op29(0):Op2A(0):Op2B(0):Op2C(4):Op2D(6):Op2E(7):Op2F(38):Op30(15):Op31(0):Op32(0):Op33(0):Op34(0):Op35(43):
Op36(0):Op37(0):Op38(41):Op39(0):Op3A(0):Op3B(0):Op3C(0):Op3D(0):Op3E(0):Op3F(0)

Oopho2ei

4th September 2008, 00:07

You recorded them in little endian, did you?
yes.

I didn't find anything usefull.
I also recorded a trace of the program counter. (see above). I think that should help you.

Disabled

4th September 2008, 00:25

Thanks a lot, the PC might help. At least it reduces the possible wrong registers to two as a jump is not taken. The cmd2 trace does not help though, as there is no SUB till that point, but it might help in the future.

*edit* If you can dump the registers real quick, a dump after the 125th cmd (PC=1AB74) might be usefull...

*edit* I'm done here for today. If every trace has the same difficulty no matter what cmd you trace, please try to do the ones with a high apperance like Op 0x35 (LDW) or 0x38 (SDW) or 0x19 (ADDIE).

*edit* I hacked the code to correct the PC after it didn't jump. After that the PC and ADDs are correct for another 230 instructions, that are 20 instructions after the first trap and this time its the correct trap #110h and you already did a memdump and regdump at that time. The registers are all correct at that point, but the mem looks utterly different... wait its little endian... ok, now there are just a few instances where they differ. But a lot is written correctly to the memory so my SDW has to be ok (and no other store is used). So the error has to be somewhere else...

*edit* I did some intensive testing but didn't find anything. I tried to look at every command issued until the first error occurs. With comparisons I changed the outcome and I just deactivated some other commands. The theory was that if one cmd is wrongly implemented, if I change them the first error would still be issued at the same place (or later if I accidently corrected it). But with every cmd I tested, the error came early or I could somehow other rule that cmd out. I'm a little puzzled now, what could be the problem.
Our memdumps differ at 3 locations. The first is 0x0000 where you overwrite it with zeroes. Then there are two locations, the one is 16 bytes you overwrite with FFs the last one is 40 bytes you again overwrite with 00s thats all. I don't write to these adresses at all (I at least scanned for accesses to adress 0x0000 without success). But you stored 00s and ffs that appear pretty often, so if they go somewhere else, they probably won't produce a different mem compared to the source...

derbeDeus

4th September 2008, 14:50

In an older post you had

Is there a list of BD+ discs so i could choose which one to buy? :rolleyes:

Edit: here are some:
28 Days Later
28 Weeks Later
August Rush
Cast Away
Dan in Real Life
Die Hard 1-4
Dogma
...

I don't think
August Rush (Warner)
Dan in Real Life (Walt Disney)
Dogma (Sony)
Gattaca (Sony)
No Country for Old Men (Walt Disney)
Sleuth (2007) (Sony)
are BD+ titles. They are just titles released on March 11, 2008 together with Hitman, Independence Day and I Robot from Fox.
The only titles that have BD+ so far are the ones from Fox and MGM (but that's still Fox contribution).

ala42

4th September 2008, 15:14

The load and save commands of the post #23 simulator can not work. Big and little endian cpus use the same address order when accessing bytes, shorts and longs. Shorts and longs have a different byte order. When emulating a big endian cpu on a litte endian cpu you have two choices:
- swap the bytes for long and short accesses
- swap all longs during program load time and swap addresses for short and byte accesses. No byte swapping needed as this is already done at load time. Longs can be read as they are, shorts can be read as they are from a modified address (XOR 2), bytes can be read as they are from a modified address (XOR 3). This will work only if short and long accesses are made on word boundaries.

The post #23 simulator uses a mixture of both, byte swapping and address swaping, which can not work.

Oopho2ei

4th September 2008, 17:09

By the way, what's actually the long term goal of the BD+-VM? I'd say, bringing it to a form in which it can be easily integrated into other programs such as KenD00's DumpHD (which is by the way Java, too).
A open source reference design (written in a well known programming language) of a working emulator. I hope KenD00 will find the time to add BD+ support to DumpHD. :)

If you can dump the registers real quick, a dump after the 125th cmd (PC=1AB74) might be usefull...
ok

If every trace has the same difficulty no matter what cmd you trace, please try to do the ones with a high apperance like Op 0x35 (LDW) or 0x38 (SDW) or 0x19 (ADDIE).
I will have them soon.

The registers are all correct at that point, but the mem looks utterly different... wait its little endian... ok, now there are just a few instances where they differ. But a lot is written correctly to the memory so my SDW has to be ok (and no other store is used). So the error has to be somewhere else...
This is a memory snapshot directly before program execution starts: http://uploaded.to/?id=5rgdzs

Also i haven't studied the exception handling (of e.g. division by zero) yet. So if an exception occurs the result of an instruction is currently undefined. I assume you just ignore them?

I don't think
August Rush (Warner)
Dan in Real Life (Walt Disney)
Dogma (Sony)
Gattaca (Sony)
No Country for Old Men (Walt Disney)
Sleuth (2007) (Sony)
are BD+ titles. They are just titles released on March 11, 2008 together with Hitman, Independence Day and I Robot from Fox.
The only titles that have BD+ so far are the ones from Fox and MGM (but that's still Fox contribution).
I took the list from: http://www.avsforum.com/avs-vb/showthread.php?t=993757. Do you think BD+ will soon no longer be used?

Disabled

4th September 2008, 18:27

This is a memory snapshot directly before program execution starts: http://uploaded.to/?id=5rgdzs
Shouldn't that be exactly the first 2MB (minus header) from the 20MB file you gave us? Padded with zeroes of course.
I'm not at home right now to compare that myself.

And you are right. I don't handle exceptions and overflows and such in any way, so they are just passed from c++ to the vm.
Perhaps it would be better to invest time into that first, as it has to be done anyway and if the errors occur because of that, I will never find them unless perhaps you post the trace for every command issued.

@ala42 That VM in post #23 was never in production, it was a prototype and never ran a single line of BD+ code. But please have a look at the vm in post #60 (http://www.zshare.net/download/183268689ed30ac4/) and feel free to comment or add or restructure the code in any way you like. Sadly it is not the latest copy I have at home, but all I added was debugging code, so the VM should handle everything excactly the same way.

ala42

4th September 2008, 18:36

The vm from post #60 does the same swapping of both data and addresses, so it has the same problem.

Oopho2ei

4th September 2008, 19:00

Shouldn't that be exactly the first 2MB (minus header) from the 20MB file you gave us? Padded with zeroes of course.
I'm not at home right now to compare that myself.
It should be (except the conversion from big endian to little endian you should do). But maybe it is changed before execution. You can check that now. Making a snapshot is done very easily compared to getting those traces.

I will never find them unless perhaps you post the trace for every command issued.
I have added cmd35 and the others will probably follow until these issues are fixed. See posting #61.

Also please listen to the suggestions from ala42. This big endian/little endian conversion and addressing could be our problem.

Oopho2ei

5th September 2008, 00:06

I have looked at the first trap call "trap_110". It has 5 parameters:
trap_110(0x1A64C, 0x5550, 1, 0x5570, 7);

Parameter 1,2 and 4 are compared with the memory size (0x400000).
Parameter 3 (p3) is used to calculate the following: 192*p3+408 = 600 (258h). The result (600) is compared with 0xFFFFFFFF.
The last parameter (parameter 5) is first compared with 0xFFF10000 and then with 6. The result of all this is a status code 0x80000001 which is stored in r01. Depending on the parameter values passed the status code changes 0x80000000, 0x80000002 and others. The last parameter seems to specify which function is finally called. With those parameters as give above the selected function simply returns after a short check and both memory and register file (except r01 which takes the status code) remain unchanged.

Now any ideas what this trap could be good for? :)

Disabled

5th September 2008, 00:32

The vm from post #60 does the same swapping of both data and addresses, so it has the same problem.

Sorry I didn't answer your post accordingly the last time, Im not at home till saturday and will have access to the net very rarely.
As I understand you say there is a problem with accessing word and byte length memory are you? Till the first error occurs, there are only word length accesses to the memory, so I ignored the correctness of these commands till now.

I compared your memory dump and it is indeed different to the 0000.whatever file. A quick analysis showed two parts are different and those were exactly those I missed in the comparison before the first trap. There still seems to be a difference before the first trap that isn't in your last dump of the mem.

Your newly posted cmds traces showed the first error occured after the 40th cmd issued, but that was due to the the B3FFFFFC in my mem where your mem has a 00000000. So I ran the mem_setup.bin and then the first error occured at an interesting point:
*edit* I wrote "0x19 (ADDIE)" so you dumped 0x19 and named it 0x18, as 0x18 is the opcode of addie, so the following is wrong */edit*
76
20CA4 E0410004 SDW R2, R1, #4h
00:02:04:7C vs 00:02:04:7C
R1:1150, R2:2047C, R28:1160, R29:1EFFDC, R30:1EFFEC, R31:1F408,
77
20CA8 60420400 ADDIE R2, R2, #400h
00:00:11:5C vs 00:02:08:7C

So R2 contains the right data after the 76th cmd issued, but if I add 400 its wrong? I can't figure out how your "thing" is calculating this, but then again its late, so I'll leave this question to another one/time.
Last minute analysis: I think I at least did an error in dissassembling the addie cmd (for the screen output), I think I used the unextended immediate. But then 400 is the same in both and I'm pretty sure the vm itself used the right immediate...
Thats it for today.

ala42

5th September 2008, 01:44

Sorry I didn't answer your post accordingly the last time, Im not at home till saturday and will have access to the net very rarely.
As I understand you say there is a problem with accessing word and byte length memory are you?
Correct. As posted above, big endian and little endian cpus use the same addresses when accessing bytes, so the LB/SB handler must not change the address with xor 3. Same for the LW/SW opcodes.

Post #20 tells you to do the xor 3 (^3). The only logical reason is that the byte order is already swapped inside all 32 bit words, so this has to be corrected. When you check the memory dumps posted later you see these dumps are already swapped, as expected.

You have to decide on which data the simulator should run: Original like in the svm files or pre-swapped like in the dumps. The simulator Oopho2ei is analysing obviously uses the pre-swapped data for performance reasons running on a little endian cpu, as the word and long data do not need to be swapped each time when reading instruction words or data. The disadvantage is that byte and word addresses have to be modified, but this still might be faster.

I hope this clears up the confusion a bit :)

schluppo

5th September 2008, 02:11

There seem to be only two traps with 5 parameters:

UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);

and

UINT32 TRAP_PrivateKey(UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord, UINT32 keyID);

By the way, here's a nice list of the traps, as taken from the patent text (more details on traps are also in this text: http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070033419%22.PGNR.&OS=DN/20070033419&RS=DN/20070033419):
http://uploaded.to/?id=tb40lb

Now i think that trap 110 is TRAP_Aes:

By specification, the last parameter of TRAP_Aes (opOrKeyID) can have the values (0xFFF10000), (0xFFF10001), (0xFFF10002) or (other). And you said, it was compared to 0xFFF10000 in your trace... Maybe this call of trap 110 is executing some kind of AES-setup / AES-check?

For TRAP_PrivateKey, the last parameter seems to be 0 in most cases.

So, I recommend to read the details on TRAP_Aes and TRAP_PrivateKey in the patent text. Anyway, if it's not TRAP_Aes, then I would guess it's some new trap which is not documented in the patent text..

KenD00

5th September 2008, 02:35

I hope KenD00 will find the time to add BD+ support to DumpHD. :)

Time seems to be the minor problem this time, knowledge is the bigger one :D. I really enjoy reading this thread, very interesting stuff but honestly, assembler just isn't my language. If this progresses to something like BD+-Aligned Unit in -> Aligned Unit out i will gratefully use it ;).

:rolleyes:

Oopho2ei

5th September 2008, 08:24

A general note regarding the "traces" from posting #61 and others: If an instruction fails with an error/exception the result is omitted from the "log". So only increment the counter for commands which completed successfully.

Now i think that trap 110 is TRAP_Aes:
This may be right. To decrypt the section 2 of file 00001.svm trap #110 is used like this:

for (i=...,i<...;i++) {
... // some code
trap_110(....)
... //some code (see posting #34)
trap_230(...) // this is trap_xorblock(...)
... // some code
}

derbeDeus

5th September 2008, 09:29

I took the list from: http://www.avsforum.com/avs-vb/showthread.php?t=993757. Do you think BD+ will soon no longer be used?

No, I was just looking for a BD+ list. I saw that link too and only Fox tiles were BD+.
I'll make a separate thread to maintain an up-to-date list of BD+ titles.

And yes, 110 is TRAP_Aes :)

Disabled

5th September 2008, 11:12

Wow I feel a little stupid. First everyone seems to know more than me about the VM and then I again made a stupid error.
I asked for the trace of 0x19 (ADDIE) but ADDIE is 0x18, so you dumped 19 and namend it 18, that was the error in the previous post.
I corrected the filename and then there the cmd 0x19 didn't seem wrongly implemented.
The first error that occurs now is after an LB, so now the endianess might catch me. I will investigate and report back.
Indeed the LB loaded a wrong adress and after I changed it, the VM works flawless! I can't find a single problem until the 840th cmd is issued, that is trap 520 and after that, things go wrong...

Inventive Software

5th September 2008, 12:21

Progress so far is amazing gents, I await the results with baited breath. :)

schluppo

5th September 2008, 14:13

here's some thoughts (edit: that is thoughts as in "ideas", not thoughts as in "usable c-source") about how to implement the traps into a vm (according to oopho's trace of trap executions).

switch (trap_number) {
...
case 110: TRAP_XorBlock(r29, r29+4, r29+8);
...
case 230: r01 = TRAP_Aes(r29, r29+4, r29+8, r29+12, r29+16);
...
default: break; //trap_number unknown
}

void TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len)
{
UINT32 *d;
UINT32 *s;
memcpy(d,dst,len);
memcpy(s,src,len);
dst = d ^ s;
}

UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID)
{
// implement AES according to patent specification here
}

derbeDeus

5th September 2008, 15:33

It's amazing the quantity of errors on this thread :P I would expect a certain amount of errors, but this is pure careless.
r29 is sp, so the parameters are passed on the stack, since it is a 32-bit machine, big-endian, the distance between param addresses should be 4.
if key is a pointer, it should be at sp+12

and wtf is this:

UINT32 *d;
UINT32 *s;
memcpy(d,dst,len);
memcpy(s,src,len)

Oopho2ei

5th September 2008, 17:33

Indeed the LB loaded a wrong adress and after I changed it, the VM works flawless!
Congratulations. :)
Yes, i will look at the traps now. One question: does this work without overwriting some bytes of the image (00000.svm) with 00/FF prior execution? Or is this required and your machine only runs with the snapshot i provided (posting #67)? If those modifications (hacks :) ) are indeed required i would have to study the loader too.

I can't find a single problem until the 840th cmd is issued, that is trap 520 and after that, things go wrong...
For verification a snapshot before trap_520:
register file: http://uploaded.to/?id=z0msdc
memory snapshot: http://uploaded.to/?id=44w6lq

To continue execution after trap_520 a snapshot directly afterwards:
register file: http://uploaded.to/?id=ljl4an
memory snapshot: http://uploaded.to/?id=sdgznt

Let me know if you need more traces of instruction results. You will probably soon need a longer program counter trace.

It's amazing the quantity of errors on this thread :P I would expect a certain amount of errors, but this is pure careless.
I'm sorry about the cmd18/cmd19 logfile/trace mistake. It has been corrected in posting #61. I decided not to go to bed although i was dead tired so people would get this one day earlier. I try to provide enough redundancy (like those instruction results, memory dumps, etc) so that the reader can do error detection and correction on it's own. Again: if you find anything please leave a note here or pm me. :rolleyes:

And yes, 110 is TRAP_Aes :)
Seems to be correct. I have observed the calculation of the round key table using the aes s-box. Sadly i can't find any documentation of the return codes (like 80000001h). Did somebody else find anything?

Disabled

6th September 2008, 05:17

Sadly it doesn't work with the 00000.svm, only with the mem_setup.bin. If the first 4 bytes are zeroed as far as I can tell it works the same as the mem_setup.bin, but there are still some more bytes zeroed 0007ece4-0007ed0b to be precise.
The FF differences were due to LB not loading one FF and thus the SDW storing 00s instead of FFs.

On the plus side, if I just ignore trap 110, the VM is in exactly the same state as your dump shows right before trap 520. Same mem, same regs.
So I wonder, what are the traps 110 doing? OK, AES but on what memory are they working? Can't be on that 4MB address space the VM uses can it?
Trap 520 on the other hand changes around 300 bytes of the memory(according to your dump).

With the help of someone I was also able to correct two errors that didn't show up until now, but might have in the future.

sakman

6th September 2008, 08:34

Sadly it doesn't work with the 00000.svm, only with the mem_setup.bin. If the first 4 bytes are zeroed as far as I can tell it works the same as the mem_setup.bin, but there are still some more bytes zeroed 0007ece4-0007ed0b to be precise.

If you look at the header for 00000.SVM:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4

Coincidence?

Oopho2ei

6th September 2008, 10:38

On the plus side, if I just ignore trap 110, the VM is in exactly the same state as your dump shows right before trap 520. Same mem, same regs. So I wonder, what are the traps 110 doing? OK, AES but on what memory are they working? Can't be on that 4MB address space the VM uses can it?
As you know the traps are executed on the host system so any trap can both change the state of the host system (eg. replace some bytes in the video/audio output of the player) and the state of the guest system. I For obvious reasons i only show you the guest system.

All the trap_110 calls before trap_520 have as last parameter (UINT32 opOrKeyID) values like 5,6,7,8 which are all less than FFF10000h and are therefor according to the patent "player key" (numbers). The trap_aes code and memory is on the host system side so you don't see anything of what is going on there.

Can you check if the first access to r01 after execution of trap_110 is a read or a write access? It would be interesting to know how they evaluate the return code (eg. 80000001h).

Trap 520 on the other hand changes around 300 bytes of the memory(according to your dump).
You can check the parameters passed to trap_520 before it is called (r29 acts as stack pointer -> see posting #34). Maybe you can find the length and address of the difference in the memory snapshots in those parameters. :)

If you look at the header for 00000.SVM:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4
You are right. This is indeed interesting. I will check what code is accessing that area of the header. :thanks:

edit: i have now checked how a trap call changes the watchdog timer: every trap call resets the timer back to 7FFFFFFFh. I have corrected this in posting #25.

Disabled

6th September 2008, 14:21

These are the cmds before Trap 520:
832 19FE0 E3FD0000 SDW R31(0x00000001), R29(0x001EFF64), 0x0000
833 19FE4 E3FD0004 SDW R31(0x00000001), R29(0x001EFF64), 0x0004
834 19FE8 C0200000 LHI 0x0000
835 19FEC 642121A0 ADDI R1(0x00000000), R1(0x00000000), 0x21A0
836 19FF0 E03D0008 SDW R1(0x000021A0), R29(0x001EFF64), 0x0008
837 19FF4 603EFFF4 ADDIE R1(0x000021A0), R30(0x001EFF90), -0x000C
838 19FF8 E03D000C SDW R1(0x001EFF84), R29(0x001EFF64), 0x000C
839 19FFC B4000BFC JAL 0x0BFC
840 1ABFC E4000520 TRAP 0x0520
So it seems it has 4 arguments. The first two parameters passed have a value of 0x01, the third is the destination address where the data is overwritten, the fourth is a pointer where a return value is written. I can't find anything that relate to the length of the data changed (284 bytes plus one byte at the return address).

About Trap 110, the two times it is executed, R1 is overwritten directly after that.

*edit* I restructured the output a little (see above) so grab the new version here (http://www.zshare.net/download/183268689ed30ac4/). For first timers I packed the source, the (modified) mem file and the debug traces into one zip (http://www.zshare.net/download/182976554aff8133/). Just unzip, compile and run the output. Then you get the output trace (pipe it into a file) and dump.txt (I should change the extension) of the mem right before trap 520.
*edit2* I rewrote the dasm function, so if you find an error in the disassembled commands (like the LHI missing its register in the trace above) please give me a note.

*edit* forgot the zip

*edit2* I updated the bdvm.c in the source only link. If you download the full package, make sure you grab this version as well.

ala42

6th September 2008, 15:40

You forgot to add PC in the JAL instruction.

Disabled

6th September 2008, 15:47

Youre right, right now the jumps are decoded relative and the PC is not added. That was already on my todo list, but I forgot to mention it. Thanks for reminding me.

sakman

6th September 2008, 16:51

You are right. This is indeed interesting. I will check what code is accessing that area of the header. :thanks:

Interesting to see what is in those 28h bytes that are not copied at the end of each BDSVM_CC section. The size of the footer seems to be consistent across each of the 200000h sections. Header 18h bytes, Footer 28h bytes.

0x18+0x7ECE4+0x28=0x7ED24 end of 1st section.
Other sections: 0x18+0x1FFFC0+0x28=0x200000.

I looked at a different 00000.svm and the footer was the same size 28h bytes.

schluppo

6th September 2008, 17:48

Trap 520 should be one of these two:

UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);
UINT32 TRAP_DeviceAccess(UINT32 dev, UINT32 opID, UINT8 *buf, UINT32 *len);

These are the only two traps with four parameters which don't have pointers as first two parameters (apart from TRAP_Mediawrite which wouldn't make much sense so early in execution).

My guess is that it's TRAP_DeviceDiscovery (makes sense to first discover the device before it's accessed). With this trap it's for instance possible to get "decryption keys or other cryptographic parameters" from the device.

For TRAP_DeviceDiscovery, first parameter (1) is the device number, second parameter (1) is the question id. The answer to the question is put to adress in the third parameter (0x21A0). Furthermore:

"Calling routines should pass the maximum length (in bytes) of the buffer pointed to by buf as the location indexed by len. The value returned in len will indicate the number of bytes actually written to buf". Len points to 0x1EFF84 in the above trace (the value at this position should be around 284 / 0x11C after execution).

TRAP_DeviceAccess specification is similar, it just takes an operation number instead of a question number.

By the way: In order to be able to emulate all the traps as good as possible, it would be nice to get ahold of "trap traces". By that i mean the values of (some of) the registers and (part of) the memory directly before and after each execution of a trap with a specific numer.

On more thing: the return values of the traps 110 and 520 seem to be just status flags so far.

On another note, Oopho says, that trap 110 is called several times, whereas according to Disabled, it's just called twice. Contradiction?

chavonbravo

6th September 2008, 17:55

Seems to be correct. I have observed the calculation of the round key table using the aes s-box. Sadly i can't find any documentation of the return codes (like 80000001h). Did somebody else find anything?

Sounds like it's simply a status return value denoting that everything went ok. On patent description:

"Most of the exemplary TRAPs return a status value (e.g., a constant STATUS_OK that indicates success, or a value indicating an error condition, or that otherwise describes the result or state of the system or operation). For brevity, the contents of the return value are generally omitted in the TRAP descriptions below if the value is simply a status value. Some TRAPs do not return any status value, but may fail (e.g., if given illegal input parameters), in which case content code may need to ensure that appropriate inputs are provided and/or verify that the operation completed as expected."

On section 0130 of patent description it describes the aes trap a bit also, it seems that we'll have to find out all the aes player keys for whatever player will be emulated.

Also, to help map the correct trap calls, this will help:

"To invoke a TRAP, the caller first pushes each of the parameters (starting with the rightmost parameter in the prototype definition)"

So we can just look at the values pushed, the order, then go to patent description and find out much easier.

edit:

Though from what schluppo derived about trap 520, it seems the order of parameters pushed is really left to right in the prototype definitions?? There's no list of qID's in the description which would have helped.

Disabled

6th September 2008, 18:07

In my trace (and the PC is always correct according to the PC trace) there are only two calls to trap 110 before the first trap 520.

The address in the 4th parameter indeed contains 0x11c after the trap 520 was executed. I'm sorry I didn't get it in the first run, I think I was confused by comparing the wrong dumps.
Before the trap it contains 0x140.

Oopho2ei

6th September 2008, 21:13

Ok i have studied the path between 00000.svm and mem_setup.bin. To load the 00000.svm do the following:

1. parse the header: bytes 0x14-0x17 make up a length 'len' stored as big endian (thanks to sakman for the hint :) )
2. load the section from 0x18 to len+0x18-1 of 00000.svm into memory (and convert from big endian to little endian)
3. overwrite the first 0xFFF bytes of the loaded section with zeros
4. done. Now start execution at 0x1000

It took me a long time to figure out these simple steps. Hope it was worth it. :helpful:

Interesting to see what is in those 28h bytes that are not copied at the end of each BDSVM_CC section.
I believe it's a checksum or signature. Maybe if the signature/checksum fails the player looks at the copy in the backup directory. It currently looks like it is just being ignored.

My guess is that it's TRAP_DeviceDiscovery (makes sense to first discover the device before it's accessed).
That is my guess too. I update the table in posting #34 regularly :)

On another note, Oopho says, that trap 110 is called several times, whereas according to Disabled, it's just called twice. Contradiction?
By several times i mean more than once. It's probably called very often in relation to other traps because part of the program/data is encrypted and needs to be decrypted first. I believe we are after 840 instructions still in the initialization phase.

By the way: In order to be able to emulate all the traps as good as possible, it would be nice to get ahold of "trap traces". By that i mean the values of (some of) the registers and (part of) the memory directly before and after each execution of a trap with a specific numer.
I know but i can't just give you snapshots of the host system (with the embedded guest system). It would be easy to see from whatever player this was taken from. And i obviously can't simply take snapshots of two different players and zero out differences. Furthermore it is difficult to make and use these snapshots of the host system: the guest system is single threaded the host system is (usually) not.
I will trace the data flow through the traps and report when i found something interesting. If you need specific details to continue your research/experiments just tell me.

Anyway i have the feeling that people (none slysoft employees) like 'derbeDeus' know already very well how all these traps work. So maybe invite them for a drink. ;)

Oopho2ei

7th September 2008, 01:40

I would like to document the traps like this if nobody has a better idea. Instead of pseudocode or long confusing descriptions i would use low level C notation:

//
// UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID)
//

UINT32 TRAP_Aes_Handler(UINT8 *vm_mem, UINT32 sp) // sp = stack pointer
{
UINT32 dest, src, len, key, opOrKeyID;
UINT32 status_code;

/* read parameter */

// enforce dword alignment when reading the parameter
dest = *(vm_mem + ((sp + 0x00) & 0x3FFFFC)); // destination pointer
src = *(vm_mem + ((sp + 0x04) & 0x3FFFFC)); // source pointer
len = *(vm_mem + ((sp + 0x08) & 0x3FFFFC)); // length value
key = *(vm_mem + ((sp + 0x0C) & 0x3FFFFC)); // key pointer
opOrKeyID = *(vm_mem + ((sp + 0x10) & 0x3FFFFC)); // operation number or

status_code = TRAP_Aes(vm_mem, dst, src, len, key, opOrKeyID);

switch (status_code)
{
case 0x80FFFFFF: break; // TODO: ?
case 0x80000002: break; // TODO: ?
case 0x80000001: break; // TODO: ?
}

return status_code;
}

UINT32 TRAP_Aes(UINT8 *vm_mem, UINT32 dst, UINT32 src, UINT32 len, UINT32 key, UINT32 opOrKeyID);
{
UINT32 tmp;

tmp = 192*len+408; // store this somewhere in memory

if (dst >= 0x400000)
{
// TODO: ?
}

// TODO: ?

if (opOrKeyID >= 0xFFF10000)
{
// opOrKeyID = operation code
switch (opOrKeyID)
{
case 0xFFF10000: // TODO: AES_ECB_ENCRYPT
case 0xFFF10001: // TODO: AES_ECB_DECRYPT
default: // TODO: AES_ECB_DECRYPT_MEDIA_KEY
return 0x80000001;
}
} else {

// opOrKeyID = player key number
if (opOrKeyID > 6)
{
// TODO: ?
return 0x80000001;
} else {
// TODO: ?
}

}

}

void main()
{
UINT8 vm_mem[0x400000]; // 4MB memory
UINT32 reg[32]; // register file

reg[01] = TRAP_Aes_Handler(&vm_mem, reg[29]);

}
You should use equivalent high level constructs of your programming language.

Disabled

7th September 2008, 03:40

Ok i have studied the path between 00000.svm and mem_setup.bin. To load the 00000.svm do the following:

1. parse the header: bytes 0x14-0x17 make up a length 'len' stored as big endian (thanks to sakman for the hint :) )
2. load the section from 0x18 to len+0x18 of 00000.svm into memory (and convert from big endian to little endian)
3. overwrite the first 0xFFF bytes of the loaded section with zeros
4. done. Now start execution at 0x1000

I somehow don't think this is right. 0x14-0x17 contain the start address where to zero memory.

The header is
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B
06 00 00 00 00 07 EC E4. 0x7ece4 is the correct start address, the length is (at least) 0x28 and those bytes have to be zeroed. I can't find the length in the header, but the next bytes after 0x7ece4 are zero anyway.
I just cross checked this with the second segment, and there the start (0x14-0x17) points to the very end of the segment (0x28 bytes to end, thats the signature/footer?), so there is nothing to be zeroed.

The first 4 bytes after the header are "B3 FF FF FC" and have to be zeroed as well. I don't know if this is neccesary with other segments, as I don't have traces of those. Also it doesn't seem to me those bytes have any information in them (They decode to a J #0000, so seem to be code and not addresses...)

*edit* Does anyone have ideas how to administrate this project? I have the feeling that others could greatly improve my source but don't because they think they would interfere with my development process. Any ideas about setting up an svn or something? Sourceforge or the like? Or do you think this is a little over the top and doesn't need more than posting sources every now and then?

evdberg

7th September 2008, 09:35

I know I am nitpicking, but when in C a variable is a pointer to an object, then adding something to this variable is automatically multiplied by the sizeof the object the pointer is pointing to. In this case vm_mem++ will increase vm_mem with 4, not by 1. In other words, it's better to keep all pointers to data (vm) memory as a byte pointer.

Oopho2ei

7th September 2008, 10:59

I somehow don't think this is right. 0x14-0x17 contain the start address where to zero memory.
Those bytes are already zeroed during your memory allocation process. You don't load the full first section of 00000.svm only to overwrite the parts with zeros you shouldn't have loaded in the first place. According to my description len+1 bytes were copied so i added a '-1'. Did you mean that?

The first 4 bytes after the header are "B3 FF FF FC" and have to be zeroed as well. I don't know if this is neccesary with other segments, as I don't have traces of those. Also it doesn't seem to me those bytes have any information in them (They decode to a J #0000, so seem to be code and not addresses...) The other segments (except the first two of 000002.svm) are encrypted or just filled with random data. We currently can't see the internal structure. (remember i uploaded the beginning of (decrypted) section 2 of file 000001.svm in posting #19). For now only zero the beginning of the very first section of 00000.svm.

*edit* Does anyone have ideas how to administrate this project? I have the feeling that others could greatly improve my source but don't because they think they would interfere with my development process. Any ideas about setting up an svn or something? Sourceforge or the like? Or do you think this is a little over the top and doesn't need more than posting sources every now and then?
svn/cvs/git is really needed.

I know I am nitpicking, but when in C a variable is a pointer to an object, then adding something to this variable is automatically multiplied by the sizeof the object the pointer is pointing to. In this case vm_mem++ will increase vm_mem with 4, not by 1. In other words, it's better to keep all pointers to data (vm) memory as a byte pointer.
I declared vm_mem as UINT8 in the main() but now i see the "UINT32 *vm_mem" in the TRAP_Aes_Handler declaration. Thank you. :)
There would also be a problem if the parameters are not stored as little endian.

edit: regarding the mapping "trap_number -> trap_name". When you compare my little trap_number list in posting #34 with the list of trap_names made by schluppo: http://uploaded.to/?id=tb40lb in posting #74 you might notice that there are four 01XX traps which make up a group containing trap_aes. Now when you look at the trap_name list from schluppo you will see that he has also put trap_aes in a group of four "crypto traps". So one could think that all the other 01XX traps are crypto traps as well. The same applies to traps_02XX and traps_05XX where we already know a group member. Note that my list of trap numbers is probably not complete as i have only written down trap numbers which are used or at least referred to by the vm program on this blue ray disc i use. I will try to get a full list now.

edit: i believe return code 80FFFFFF indicates an error like usually return(-1) so maybe interpret the lower 24 bit as signed number.

edit: 000010 = void TRAP_Finished();
000020 2 parameters -> ?
000110 = UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);
000120 = UINT32 TRAP_PrivateKey(UINT32 keyID, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord);
000130 = UINT32 TRAP_Random(UINT8 *dst, UINT32 len);
000140 = UINT32 TRAP_Sha(UINT8 *dst, UINT8 *src, UINT32 len, UINT32 op);
000210 = UINT32 TRAP_AddWithCarry(UINT32 *dst, UINT32 *src, UINT32 len);
000220 = UINT32 TRAP_MultiplyWithRipple(UINT32 *dst, UINT32 *src, UINT32 len, UINT32 multiplicand);
000230 = UINT32 TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len);
000310 = UINT32 TRAP_Memmove(UINT8 *dst, UINT8 *src, UINT32 len);
000320 = UINT32 TRAP_MemSearch(UINT8 *Region, UINT32 RegionLen, UINT8 *SearchData, UINT32 SearchDataLen, UINT32 *Dst);
000330 = UINT32 TRAP_Memset(UINT8 *dst, UINT8 fillvalue, UINT32 len);
000410 = UINT32 TRAP-SlotAttach(UINT32 slot, UINT32 codeLen);
000420 = UINT32 TRAP-SlotRead(UINT8 *dst, UINT32 slot);
000430 = UINT32 TRAP_SlotWrite(UINT8 *newContents);
000510 = UINT32 TRAP_DeviceAccess(UINT32 dev, UINT32 opID, UINT8 *buf);
000520 = UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);
000530 = UINT32 TRAP_DiscoveryRAM(UINT32 unknown, UINT32 dst, UINT32 len);
000540 = UINT32 TRAP_MediaReadFile(UINT8 *FileName, UINT32 SectionNumber, UINT32 Unknown, UINT32 *len, UINT8 *dst);
000550 = UINT32 TRAP_MediaSHAFileHash(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst);
000560 = UINT32 TRAP_RunNative(UINT8 *signature, UINT32 sigLen, UINT8 *code, UINT32 codeLen);
000570 ? nop/vendor specific?
008010 = UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
008020 ?
008030 ?
Note this is a guess. I would have to verify this. But this doesn't seem to be coincidence. :)

Disabled

8th September 2008, 16:32

the parts with zeros you shouldn't have loaded in the first place. According to my description len+1 bytes were copied so i added a '-1'. Did you mean that?

It took some time (that I don't have much right now) and a few rereads to understand your description. Everything after the address in 0x14-0x17 is zeroed (ie. not loaded), so I just load from 0x18 and stop at the address in the header (+0x18-1).

Apart from that and a few technical things, is there anything I can implement in the VM right now? I could do a few traps that seem to not need any further documentation like TRAP_XorBlock, TRAP_Memmove, TRAP_Memset, TRAP_Random and TRAP_Memsearch or TRAP_Aes perhaps. I was thinking about using this (http://www.lomont.org/Software/Misc/AES/AES.php) lib for AES decryption or does anyone have better ideas? (Of course TRAP_Aes isn't fully documented, but the main de/encryption could be implemented).

Oopho2ei

8th September 2008, 17:20

It took some time (that I don't have much right now) and a few rereads to understand your description. Everything after the address in 0x14-0x17 is zeroed (ie. not loaded), so I just load from 0x18 and stop at the address in the header (+0x18-1).
again as sample C code:

0. some declarations:
UNIT32 mem_size = 0x400000;
UNIT32 len;
UNIT8 *p;

1. UNIT8 vm_mem[mem_size]; // allocate memory

2. for (i=0; i<vm_mem; i++) vm_mem[i] = 0; // zero memory
3. // open file 00000.svm and let p be a pointer pointing to the beginning of the file
4. // len = bytes 0x14-0x17 interpreted as UINT32 in big endian
5. for (i=0; i<len; i++) vm_mem[i ^ 0x03] = *(p + 0x18 + i) // load the section (as little endian)
6. // close file
7. for (i=0; i<0x1000; i++) vm_mem[i] = 0; // erase the first 0x1000 bytes

8. now zero all registers, set watchdog to 0x7FFFFFFF, pc to 0x1000 and start execution.
In general if something is unclear just ask for an example. It is hard to see where you and other people get confused. :confused:

Apart from that and a few technical things, is there anything I can implement in the VM right now? I could do a few traps that seem to not need any further documentation like TRAP_XorBlock, TRAP_Memmove, TRAP_Memset, TRAP_Random and TRAP_Memsearch or TRAP_Aes perhaps. I was thinking about using this (http://www.lomont.org/Software/Misc/AES/AES.php) lib for AES decryption or does anyone have better ideas? (Of course TRAP_Aes isn't fully documented, but the main de/encryption could be implemented).
Yes please start implementing these traps. Maybe use a separate file for every trap group (crypto traps, mem traps, ...).

Has anyone setup an sourceforge account for this project yet? Otherwise i would create one. :rolleyes:

Disabled

8th September 2008, 17:52

again as sample C code:

7. for (i=0; i<0x1000; i++) vm_mem[i] = 0; // erase the first 0x1000 bytes

Thats a new information is it? And this has to be done for every section? I remember you writing to only erase the beginning in the first section?!
Yes please start implementing these traps. Maybe use a separate file for every trap group (crypto traps, mem traps, ...).

Will do if I find the time.
Has anyone setup an sourceforge account for this project yet? Otherwise i would create one. :rolleyes:
I don't think so, please do if you find the time. I hope this will ease the project administration and source distribution. I feel there are many brilliant brains watching this thread but not giving as much input as they could, perhaps because they don't want to be contributing to a project that might provide problems with their local laws...

ala42

8th September 2008, 17:57

I was thinking about using this (http://www.lomont.org/Software/Misc/AES/AES.php) lib for AES decryption or does anyone have better ideas?
Better take the AES implementation used in AACS keys (http://forum.doom9.org/showthread.php?t=123311)
Has anyone setup an sourceforge account for this project yet? Otherwise i would create one. :rolleyes:
That is not worth the trouble, as it will be shut down quite soon.

Disabled

8th September 2008, 18:12

Better take the AES implementation used in AACS keys (http://forum.doom9.org/showthread.php?t=123311)

Does it have AES encryption? Or is it never used?

That is not worth the trouble, as it will be shut down quite soon.
Any other suggestions? Or just stick to the code - upload sheme?

ala42

8th September 2008, 18:40

Does it have AES encryption? Or is it never used?

Sorry, I do not know the crypto stuff details.

Disabled

8th September 2008, 19:59

Sorry, I do not know the crypto stuff details.

case 0xFFF10000: // TODO: AES_ECB_ENCRYPT
case 0xFFF10001: // TODO: AES_ECB_DECRYPT

So I thought it is at least in the standard...
And aacs_aes.h contains the following funtions:
calculate_subdev_proc_key
decrypt_c_value
decrypt_verification_data
calculate_volume_unique_key
decrypt_title_key
calculate_title_key_file_mac
calculate_title_key_file_hash
calculate_volume_id_mac
So... I might rip the decrypt_title_key if its doing "AES_ECB_DECRYPT_MEDIA_KEY" Oopho2ei refers to and evaluate later how I do the encryption. To be honest, even if TRAP_Aes are the first two traps issued, I'll first do the others, as I'm lazy and those look easier to do.

Oopho2ei

8th September 2008, 20:11

That is not worth the trouble, as it will be shut down quite soon.
I have setup and account now. If you want to join our team please send me a private message for details. I will choose who gets access to the repository.

So... I might rip the decrypt_title_key if its doing "AES_ECB_DECRYPT_MEDIA_KEY" Oopho2ei refers to and evaluate later how I do the encryption. To be honest, even if TRAP_Aes are the first two traps issued, I'll first do the others, as I'm lazy and those look easier to do.
Leave Trap_AES for now. I haven't even described it yet. :rolleyes:

LoRd_MuldeR

8th September 2008, 22:44

Any other suggestions? Or just stick to the code - upload sheme?

Create a new SVN repository at https://opensvn.csie.org/ :)

KenD00

8th September 2008, 23:40

So... I might rip the decrypt_title_key if its doing "AES_ECB_DECRYPT_MEDIA_KEY"

aacskeys uses OpenSSL for AES, the methods in aacs_aes perform special tasks and are not general purpose AES functions. If you need AES only take a look at the good implementation from Brian Gladman (http://fp.gladman.plus.com/AES/index.htm).

:rolleyes:

NeonMan

9th September 2008, 21:16

...That is not worth the trouble, as it will be shut down quite soon.

Sourceforge is known to shut down questionable projects, I think playfair received a DMCA and sourceforge removed all the project. I suggest launchpad (https://launchpad.net/) but the best option is a personal server like zotty did with the defunct? decryptHD.
I know this is a quite delayed opinion but I couldn't find the time to write it and I think a CVS/SVN will help the development of this BD+ implementation and will also allow a faster error tracking.

Disabled

9th September 2008, 22:07

I'll just quote Oopho2ei:
I have setup and account now. If you want to join our team please send me a private message for details. I will choose who gets access to the repository.

And about the AES implementation:

Leave Trap_AES for now. I haven't even described it yet. :rolleyes:

Thus: Everyone who wants to help, just write him a message about the details. And we'll concentrate on the easy traps for a start until TRAP_AES and others are documented.

The next feature I implemented was an .svm parser. Now you can load a segment of an svm file, the BDSVM_CC header is verified, the length is calculated and then the data is copied to mem and the first 0x1000 bytes are zeroed.
And I again rewrote the dasm part, as It was a horror to debug both earlier versions. I hope thats the last time I'll do that.

Next stop: implementing traps
*edit* perhaps I'll restructure the code first...
*edit2* oh and perhaps I'll also rewrite the debug trace functions first...
restructuring the code is also a good thing, so more ppl can work on the code without the need of backporting or merging code.

Oopho2ei

9th September 2008, 23:47

Thus: Everyone who wants to help, just write him a message about the details. And we'll concentrate on the easy traps for a start until TRAP_AES and others are documented.
I don't hide the details. If i knew i would post them (except player specific stuff or course). Of course you can ask me and depending on your question(s) i will take a look.

The next feature I implemented was an .svm parser. Now you can load a segment of an svm file, the BDSVM_CC header is verified, the length is calculated and then the data is copied to mem and the first 0x1000 bytes are zeroed.
And I again rewrote the dasm part, as It was a horror to debug both earlier versions. I hope thats the last time I'll do that.
You are doing well. It's good to have you in the team. :)

For those who want to play around a bit with the emulator here is an extended program counter trace: http://uploaded.to/?id=gx84gh (~4 million entries)
If you need other (extended) traces please leave a note here or send me a private message.

Thanks to all who made suggestions where to host our software development project. We currently have a working repository but this is still very helpful if we get trouble and have to move to another location.

KJI

10th September 2008, 00:08

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.

Oopho2ei

10th September 2008, 16:22

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.
The code looks very clean and well structured. I like it but i suggest to convert the image (section from 00000.svm) from big to little endian before execution. This should be more efficient compared to converting the data/code on every access.

Btw. if you want to test a trap implementation use the pc trace from posting #110 and ask me for some memory snapshots. You can also send me a modified memory snapshot so you can run your own program with your own data on a fully implemented player. I would need the initial register values and where to start and stop execution. But please comment your program and all the other changes you made so i can verify that no player specific data is retrieved. You would get the result as a memory/register file snapshot.

loric

10th September 2008, 18:16

The code looks very clean and well structured. I like it but i suggest to convert the image (section from 00000.svm) from big to little endian before execution. This should be more efficient compared to converting the data/code on every access.

I beg to differ. In case of self-modifying code (and in the future bd+ may become that), what you suggest would break things. My suggestion is to behave as close as possible as the real BD+VM which, AFAIR, doesn't do any endianess conversion at load time.

BTW, great job.

Oopho2ei

10th September 2008, 19:39

I beg to differ. In case of self-modifying code (and in the future bd+ may become that), what you suggest would break things. My suggestion is to behave as close as possible as the real BD+VM which, AFAIR, doesn't do any endianess conversion at load time.
Please elaborate on this seĺf modifying code problem you see. If all memory accesses of instructions and traps are adapted according to our memory format (e.g. big endian or little endian) it's impossible for the program running on the vm to tell any difference. We could even store our instructions in some weird looking sql database tables. As long as everything is correctly implemented our memory organisation is fully transparent to the program which is running on our emulator. But what does change is the performance of our emulator.

Edit: The following archive contains a series of 16 snapshots taken after each of the first 16 trap executions. Example: post_trap_mem_000.bin + post_trap_reg_000.bin is a snapshot after the execution of the first trap which is Trap_AES (TRAP #110). Each snapshot is supposed to be loaded by your emulators with every new trap call you encounter when executing the given program (see the program counter trace in posting #110). In other words instead of executing the trap you only load the corresponding snapshot. After the 11th trap call you will find yourself right in the middle of the decryption process of the second section from the second file (00001.svm). Also there seem to be pictures in the *.svm files somewhere. So it's getting more interesting soon. :)

post trap snapshots: obsolete. see posting #130

(for those who don't know: we are executing the first section of http://uploaded.to/?id=jqstc8 which is one of the files stored in the BDSVM directory of every BD+ protected blue ray disc)

sakman

11th September 2008, 09:42

Looks like a date is also part of the header:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4

07D7h=2007
0Bh=11 (November)
06h=6

bmnot

11th September 2008, 10:11

ok... sorry for the delay.

It's interesting how things evolved (nice work!).

As announced in my last post, I implemented a Java BDVM together with a graphical debugger.
It executes the BD+ program correctly until the 0x520 trap (memory dumps and registers match) - just as the other two VMs do, which I used as an orientation/prototype. The code is pretty similar (in my notion, it is slightly cleaner and more compact than my C implementation due to Java's convenient ByteBuffer classes which take care of endianness; Apart from that, the graphical Debugger is actually less code than the VM :) (294 SLOC vs. 245 SLOC) ; the implementation is big endian throughout; I just started looking at the traps).
I don't know whether this will be useful. I mainly implemented this for fun and educational purposes (for me and maybe others).

I agree that the implementation by KJI's friend is very clean (this probably doesn't matter, but I think 'vm->watchdog-=2;' in 'void ExecuteInstruction(BDVM *vm)' is not correct - see post #25).

Below is a screenshot of the debugger (ok, no screenshot... no idea where to upload it to.. it's included in the zip). BTW, the disassembling is dynamic (updated after every step).

Source (2 Java files - 1 VM and 1 Debugger) is available from here: http://uploaded.to/?id=s46hrd

derbeDeus

11th September 2008, 11:24

Looks like a date is also part of the header:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4

07D7h=2007
0Bh=11 (November)
06h=6

The BDSVM block format is:
BDSVM_header
-----8<------
magic 8 bytes - is string 'BDSVM_CC'
?version 1 byte - always 1?
?unk9 4 bytes - usualy 1 (sometimes 0x2000 in AVP2, Jumper)
year 2 bytes
month 1 byte
day 6 bits
chunkSize 26 bits - always 0? (don't remember why I named it)
datalen 4 bytes - size of code chunk following the header
----->8------
ContentCodeBlock of datalen bytes
CC_signature 40 bytes

that's padded with 0's to 2MB boundary

Oopho2ei

11th September 2008, 18:44

As announced in my last post, I implemented a Java BDVM together with a graphical debugger.
Looks awesome! Could you write a small howto to get it all working? My first attempts failed and i would like to try it :)
javac Debugger.java BDVM.java
Debugger.java:110: error: The method actionPerformed(ActionEvent) of type Debugger must override a superclass method
public void actionPerformed(ActionEvent e) {
(javac invokes gcj-4.3 here.)

Could you also please load a snapshot for traps which are currently not implemented? See postings #114 for details.

sakman

11th September 2008, 19:49

The BDSVM block format is:
BDSVM_header
-----8<------
magic 8 bytes - is string 'BDSVM_CC'
?version 1 byte - always 1?
?unk9 4 bytes - usualy 1 (sometimes 0x2000 in AVP2, Jumper)
year 2 bytes
month 1 byte
day 6 bits
chunkSize 26 bits - always 0? (don't remember why I named it)
datalen 4 bytes - size of code chunk following the header
----->8------
ContentCodeBlock of datalen bytes
CC_signature 40 bytes

that's padded with 0's to 2MB boundary

Thanks derbeDeus. Matches what I saw looking at several .svm files from different discs. I did notice that the 4 bytes of "?unk9" changed to 2000h with that bunch of March/April 2008 discs (Hitman, Jumper, Alvin, etc.). Interesting that those discs also broke the initial BD+ support in AnyDVDHD.

bmnot

11th September 2008, 21:57

Okay, I made things a little simpler:
The binaries, a readme and an ant build script are included in this updated version: http://uploaded.to/?id=khplo8
Java 1.5 or newer is required in order to compile and run the Debugger (not sure whether gcj will work).
Ant is helpful for compiling.

Regarding the traces...
Executing with the help of the dumps was successful :cool:
I'm not sure whether this is already clear: r1 seems to contain the return values from the trap calls.
Apart from r1, the registers were always identical to the dumps (if I haven't missed something).
Note that loading the mem dumps and register dumps is not automated in the debugger. Currently this has to be done manually.

The Traps were: (<dump no>) <trap id> <instruction counter>
(0) 110
(1) 110
(2) 520
(3) 520
(4) 520
(5) 110
(6) 110
(7) 110
(8) 540 4527
(9) 110 13536
(10) 230 13547
(11) 110 22408
(12) 230 22419
(13) 110 31289
(14) 230 31300
(15) 110 40161
((16) 230 40172)

Oopho2ei

11th September 2008, 21:59

Some notes about how to implement Trap_AES(0x001EF748, 0x001EF748, 0x00000080, 0x00016764, 0xFFF10001) and similar.

Remember the declaration: UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);

The last parameter (opOrKeyID) is 0xFFF10001 so we have to decrypt [len] 128bit blocks of [*src] using ECB mode with the 16-byte key [*key] and write the results to [*dst]. In ECB mode every 128 bit block is decrypted independently from all other blocks so i will only demonstrate the decryption steps for a single block.

Example snapshots: http://uploaded.to/?id=7sz6pu
This archive contains two snapshots. One was taken directly before the Trap_AES call we are looking at and one directly afterwards. You can use this to verify your implementation.

Let's look at address 0x001EF748 to find the first data block we have to decrypt and at address 0x00016764 we will find the key we have to use:
data: c5 ed 4c 67 e4 45 ca e9 40 e0 b7 89 3a c9 54 a6
key : e6 d3 0c b1 d4 29 4c 88 e4 66 29 06 34 35 25 be

Our memory content has been converted from big endian to little endian which breaks the decryption process using default aes implementations but we can simply revert the conversion:
data: 67 4c ed c5 e9 ca 45 e4 89 b7 e0 40 a6 54 c9 3a
key : b1 0c d3 e6 88 4c 29 d4 06 29 66 e4 be 25 35 34

We apply this to all 80h blocks (16 bytes each) we have to decrypt and then pass the result to a default aes-128-ecb implementation. In this example I simply put the first block in a file source.bin and called openssl from the command line:
openssl aes-128-ecb -d -K b10cd3e6884c29d4062966e4be253534 -iv 00000000000000000000000000000000 -in source.bin -out result.bin

The result is: 9d df 63 6e 37 cf b6 c6 dc b0 2d c3 a3 ca a7 f1
Now all that's left to do is to convert this block from big endian to little endian again and write the result to 0x001EF748. You can verify these steps by looking at the two snapshots i posted above. If the decryption completed without any errors we return 0 (return values are written into r01)

Okay, I made things a little simpler:
The binaries, a readme and an ant build script are included in this updated version:
I got it running!! :)

Some suggestions:
- for branches you better show the absolute address and the not the relative address
- could you automatically load the snapshots? :)
- add a data view window (maybe double the number of columns for the register view and cut its height by half to make space for a small data view window below)
- reset the watchdog timer with ever trap call to 0x7FFFFFFF
- display the parameters of the trap call directly before it is executed (e.g. by commenting the values in the stack window)
- support of data breakpoints
- color the registers red which have been changed since the last step/run
- maybe assign some hotkeys

Again good work! :)

derbeDeus

12th September 2008, 10:09

Thanks derbeDeus. Matches what I saw looking at several .svm files from different discs. I did notice that the 4 bytes of "?unk9" changed to 2000h with that bunch of March/April 2008 discs (Hitman, Jumper, Alvin, etc.). Interesting that those discs also broke the initial BD+ support in AnyDVDHD.

I think the change was later, with Alvin and AVP2 (as they also changed to MKBv7).
In my copy of Hitman it is 1 (!?)

As a side note, Fox has still MKBv7 for its titles, while others (Lionsgate) changed to 8 and 9. Probably they trust their BD+ code more than AACS protection. duh :p

gioowe

12th September 2008, 15:40

Maybe someone would like to implement the following trap:

UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
It's the trap number 0x8010.

It seems to just log debug information from the BD+ VM to somewhere.
txt is a pointer within VM space to a character sequence (single byte) with length len (not null terminated).

Maybe the BD+ implementations use it and we'll see them send some information :)

Oopho2ei

12th September 2008, 17:29

More and more people keep asking the same question about why a particular trap call didn't change anything in the guest system (that 4MB memory + registers you see).
The answer: The traps are executed on the host system side in which the guest system is 'embedded'. So a trap can make changes in the host system (e.g. replace some bytes in the audio or video stream) which are invisible to you because i only show you the embedded guest system for obvious reasons. So when you don't see any changes in the snapshots before and after a trap execution the trap can still have made changes in the host system.

UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
It's the trap number 0x8010.

It seems to just log debug information from the BD+ VM to somewhere.
txt is a pointer within VM space to a character sequence (single byte) with length len (not null terminated).
This would be an example of such a trap. It changes the host system but not the guest system. Thank you gioowe for sharing this information. :)

Edit: schluppo has found a mistake in the post_trap_snapshot archive from posting #114 (post_trap_mem_002.bin and post_trap_reg_002.bin were wrong). I have recorded a second independent series of snapshots and replaced the link accordingly. Note that starting with post_trap_mem_004.bin the two series differ at address 0x4514 and 0x4518-0x451B so the third call of TRAP_DeviceDiscovery seems to introduce random data? Maybe someone could check who is writing to these locations. We need data breakpoints too, bmnot :)

schluppo

12th September 2008, 22:00

Here are some more details on the traps :

instruction #00417:

trap_110(0x1A64C,0x5550,1,0x5570,7)

returns 0x80000001

no memory / registers changed

instruction #00475:

trap_110(0x1A64C,0x5550,1,0x5560,8)

returns 0x80000001

no memory / registers changed

instruction #00839:

trap_520(1,1,0x21A0,0x1EFF84)

returns 0

writes 284/0x11C bytes to 0x21A0 (these 284 bytes seem to contain around 84 bytes header).
the last parameter holds available / written length.
second parameter is the question-id or op-id.
maximum reply length for questions / ops #1 and #2 is 0x140, for question /op #3, it's 0x1000.

instruction #02319:

trap_520(1,2,0x2358,0x1EFF84)

returns 0

writes 292/0x124 bytes to 0x2358 (these 292 bytes seem to contain around 88 bytes header).

instruction #03539:

trap_520(1,3,0x4514,0x1EFF84)

returns 0

has 4096 bytes reserved (as 0x00) at 0x4514, writes only 60/0x3C bytes to 0x4514.

instruction #04289:

trap_110(0x156DC,0x156DC,3,0x571C,4)

returns 0

writes 48 bytes to 0x156DC.

instruction #04360:

trap_110(0x17764,0x17764,2,0x16764,5)

returns 0

writes 32 bytes to 0x17764.

instruction #04435:

trap_110(0x16764,0x56B0,1,0x16764,5)

returns 0

writes 16 bytes to 0x16764.

instruction #04527:

trap_540(0x1B428,1,0,0x1EFF54,0x22060)

returns 0

this trap writes 0x74F9C bytes to 0x22060. fourth parameter adress contains number of written bytes, fifth parameter is the dst adress.

I could not find any documented trap in the patent text that would fit to trap_540.

Decryption of the result of trap_540 happens by the following sequences of trap_110 and trap_230. Blocks of 2kb are decrypted by each iteration:

instruction #13536:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #13547:

trap_230(0x22060,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x22060.

instruction #22408:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #22419:

trap_230(0x22860,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x22860.

instruction #31289:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #31300:

trap_230(0x23060,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x23060.

instruction #40161:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #40172:

trap_230(0x23860,0x1EF748,0x200)

returns 0?

writes 2048 (0x200 times 4) bytes to 0x23860?

This finishes the first 16 traps.

Oopho2ei

12th September 2008, 22:22

instruction #04527:

trap_540(0x1B428,1,0,0x1EFF54,0x22060)

returns 0

the snapshots indicate that this trap changes 477301 bytes of the vm memory, so i guessed that the snapshot is bad and stopped here.

I could not find any documented trap in the patent text that would fit to trap_540.
Sorry, I forgot to upload the 00001.svm. This trap (trap_540) seems to load the second section of the second file (00001.svm) at the address given by the last parameter (0x22060). The section is being decrypted later (see posting #114)

Here it is (finally): http://uploaded.to/?id=tn6kle

Thanks for reminding me. :thanks:

schluppo

12th September 2008, 23:08

The difference in your snapshot series appears since these bytes hold a timestamp.

0C09D807 2D353013 => 12-09-2008 19:48:53:45
0A09D807 550F0014 => 10-09-2008 20:00:15:85

Edit (answering to Oopho):

-i.e.
0x0C=12, 0x09=09, 0x07D8=2008
0x13=19, 0x30=48, 0x35=53, 0x2D=45
0x0A=10, 0x09=09, 0x07D8=2008
0x14=20, 0x00=00, 0x0F=15, 0x55=85

-Further, it's hard, to make out a pattern in the outputs of the calls of TRAP_DeviceDiscovery(..). In the case, that the question has an answer of max-length 0x140, the output seems to have a header of around 90 bytes containing (each word of length 32 bits, little endian, parameters are _not_ identical to - but probably depend on - the trap parameters):

<length_of_result>, 0x100010, 0x4, param2, param3, param4, ~16 times: 0x00000000, <manufacturing_date>

The ~200 bytes following the header are incomprehensible to me without any hint of interpretation. Anyway, they are the 'answer' to the 'question' asked to the device.

I'd say that more calls of this trap need to be snapshot in order to get a better base for estimations.

-More details on the traps here: http://uploaded.to/?id=xqibd0

Edit: updated text file with info on all the first 16 traps.

Oopho2ei

13th September 2008, 11:00

Thank you. Were you able to trace the key from posting #121 back to a trap result?

I'd say that more calls of this trap need to be snapshot in order to get a better base for estimations
I will try to record a long series of snapshots automatically now. :)

In general for those traps you have already implemented you should still load the corresponding snapshot for verification. If yours and mine don't match you probably made a mistake. :p I provide these snapshots primarily for you to test your implementations. But they of course also useful to study traps which are currently unknown.

schluppo

13th September 2008, 14:37

The key from post #121 is the result of this trap:

instruction #04435:

trap_110(0x16764,0x56B0,1,0x16764,5)

returns 0

writes these 16 bytes to 0x16764:

E6D30CB1 D4294C88 E4662906 343525BE

trap_110/TRAP_Aes is called three times in a row, first returning 48, then 32 and finally 16 bytes, the calls are:

trap_110(0x156DC,0x156DC,3,0x571C,4)
...
71 non-trap instructions
...
trap_110(0x17764,0x17764,2,0x16764,5)
...
75 non-trap instructions
...
trap_110(0x16764,0x56B0,1,0x16764,5)

Oopho2ei

14th September 2008, 00:20

These are the first 5000 snapshots. Each snapshot was taken automatically after every trap execution. I have made multiple independent packages you can download.

Warning: You may not mix snapshots from different series therefor i have removed the link to the old snapshot series in posting #114. If you have already implemented Trap_DeviceDiscovery you have to set the timestamp according to the snapshot series you use.

Attention: All the snapshots are little endian (swapped for performance reasons) but the DLX is a big endian processor. So depending on your implementation might have to swap the byte order when loading a snapshot.

Note:: Starting with package version v1.03 we use the new diffarchive format to store the snapshots. A sample implementation can be found here (http://svn.assembla.com/svn/bdplus/trunk/diffarchive/).

current packages:
Package (v1.03) for "The Day After Tomorrow": http://uploaded.to/?id=5dwn30
Package (v1.02) for "I Robot": http://uploaded.to/?id=svpn7q

old/deprecated packages:
Package (v1.02) for "The Day After Tomorrow": http://uploaded.to/?id=g93e2g
Package (v1.01) for "The Day After Tomorrow": http://uploaded.to/?id=yodttu
Package (v1.00) for "I Robot": http://uploaded.to/?id=mej65b

Current package contents:
/
AACS/
post_trap_snapshots/
post_break_snapshots/
BDSVM/
traces/
aacskeys_output.txt
title.txt
version.txt

How to use these snapshots:
- implement a counter 'cnt' (or any other name) which is incremented after every execution of cmd39. The initial value of the counter is 0.
- load the files with filenames [snapshot_reg] and [snapshot_mem] with sprintf(snapshot_reg,"post_trap_reg_%03d.bin",cnt); sprintf(snapshot_mem,"post_trap_mem_%03d.bin",cnt); from the trap_snapshots directory
- after your implementation of the trap terminates compare your memory and register file with the snapshot you have loaded
- if your memory and register file contents doesn't match the provided snapshot then print a warning (or whatever you think is appropriate) and overwrite your register file and memory contents with the snapshot and continue execution
- use the post-break snapshots in the same way when the watchdog timer reaches zero (or becomes negative)

Benefits:
- automatic testing of your implementation on a real world example
- program execution up to the point where video/audio playback starts
- black box analysis of currently unknown traps by looking at input and output

Oopho2ei

14th September 2008, 21:38

Another mistake i made was found: cmd3A is INSTF so it sets the instruction filter value and not the watchdog timer. I have corrected this immediately in posting #27. For a short explanation of what INSTF does please see posting #44

Thanks for reporting this! :thanks:

schluppo

15th September 2008, 00:52

Here is an edited version of bmnot's quite usefull java debugger (I apologize in advance in case that anyone is offended by my bad coding style or use of bmnot's ideas):

http://uploaded.to/?id=ph017s

Things I changed:

- Treatment of parts of trap_110, treatment of trap_520.
- No treatment of actual AES functionality of trap_110 (didn't have time to implement this in java yet). No treatment of trap_540 and trap_230.
- Automatic input of "post_trap_mem_xxx" and "post_trap_reg_xxx" when untreated traps appear (Note that all the snapshots need to be located in the working dir - i.e. the dir containing the .jar - at this point).
- I added a "trap counter" to the debug-gui.
- After pressing "run", the VM will run till the 472th trap is reached (or till PC goes out of sync or breakpoint is reached).
- After that, pressing "run" will run till the next trap is reached (or till PC goes out of sync or breakpoint is reached).
- Whenever the memory is being dumped manually, a binary statistic of trap calls (trap_calls.bin) is written to the working dir.

Things to do:

- Further treatment of traps.
- Comparison of snapshots with results of treated traps.
- (Separate option to create trap-calls statistic.)
- (Separate option to select source dir for automatic snapshot input.)

Things learned:

- The debugger is quite slow on inputting PC-traces. 4,000,000 PC values take 10min+.
- The first ~2,080,000 instructions of the BD+-VM contain 477 trap calls:

239 times trap_110
234 times trap_230
003 times trap_520
001 times trap_540

- The first 477 traps do the following: 1) Setup the device 2) get some basic info on the device 3) let the device decrypt a key (taken from the .svm or from the device?) 4) load ~500kb of data from somewhere. 5) decrypt this data using the key from step 3) and also using the device's XOR-capabilities (?).
- Using the snapshots, the PC is not getting out of sync in the first 2 million instructions, indicating that the debugger's VM is quite correct (apart from the traps).

Some questions:

- What exactly does trap_540 do? I couldn't find the byte-sequences that it writes to the VM's memory anywhere in 00001.svm. Is it XORing the data from 00001.svm with the first parameter's value (0x30303030) before writing it to the VM's memory?
- Are you sure, trap_230 is pure XOR? The snapshots indicate that trap_230 is sometimes changing more than len (third parameter) times 4 byte (could also be some mistake in the snapshots / my calculations).

schluppo

15th September 2008, 04:05

The traps from the previous post are followed by:

trap #477 at ic: 2081538 is trap_530(0,0x2514,0x2000); returns 0;
writes 0x2000 bytes to 0x2514.
trap #478: trap_110(0x24FC,0x7D360,1,0x7D350,6);
writes 16 bytes to 0x24FC.
trap #479: trap_110(0x24EC,0x7D360,1,0x7D350,5);
writes 16 bytes to 0x24EC.
...
trap #484: trap_110(0x249C,0x7D360,1,0x7D350,0);
writes 16 bytes to 0x249C.
trap #485: trap_110(0x24FC,0x7D360,1,0x7D350,6);
writes 16 bytes to 0x24FC.
trap #486: trap_110(0x24EC,0x7D360,1,0x7D350,5);
writes 16 bytes to 0x24EC.
...
trap #491: trap_110(0x249C,0x7D360,1,0x7D350,0);
writes 16 bytes to 0x249C.
trap #492 at ic: 2687589 is trap_520(1,2,0x1EFA94,0x1EF890->0x190); returns 0;
identical behaviour as previous trap_520 with same params :)
trap #493 at ic: 2687599 is trap_140(0x1ef894,0x1efa94,7c,3); returns 0;
writes 0x7c bytes to 0x1ef894. this seems indeed to be trap_sha.

After trap #493, PC goes out of sync for me even when loading post_493 snapshots. The jump (BEQZ r6, 0x28) at 0x95064 (ic: 2691187) is not taken, but should be taken according to the PC-trace (r6 = 0xFF in my run).

Oopho2ei

15th September 2008, 08:04

- What exactly does trap_540 do? I couldn't find the byte-sequences that it writes to the VM's memory anywhere in 00001.svm. Is it XORing the data from 00001.svm with the first parameter's value (0x30303030) before writing it to the VM's memory?
Trap_540 is UINT32 TRAP_MediaReadFile(UINT8 *FileName, UINT32 SectionNumber, UINT32 Unknown, UINT32 *len, UINT8 *dst);

It reads [len] bytes(?) of stection [SectionNumber] from BDSVM/[FileName].svm and writes them to [dst].

Are you sure, trap_230 is pure XOR? The snapshots indicate that trap_230 is sometimes changing more than len (third parameter) times 4 byte (could also be some mistake in the snapshots / my calculations).
Yes it's pure Trap_Xor but i am currently not sure about the length.

The jump (BEQZ r6, 0x28) at 0x95064 (ic: 2691187) is not taken, but should be taken according to the PC-trace (r6 = 0xFF in my run).
Then you have a bug in your command dispatcher or and in at least one of the command handlers. Try to trace back why r06 is not zero. Btw I have heard reports of people who are already using snapshot number i (i>2000).

Edit: (removed)

Oopho2ei

15th September 2008, 18:36

The problem is the pc trace wasn't recorded at the same time as the snapshots. Through the traps "random" data is injected into the guest system which ultimately influences the instruction flow like you have experienced. So if that happens you either have to resynchronize your offset in the pc_trace.bin using the next snapshot data or simply don't use the pc trace at all. I haven't expected such non-deterministic behavior.

To provide evidence for this theory i made several pc traces and compared them. They all differ at offsets 01922478h, 019952D8h and higher. Please check if this matches your results.

I am sorry for all the confusion. It was my mistake (again) :rolleyes:

schluppo

16th September 2008, 01:23

I finished implementation of trap_230 (Trap_XorBlock) and trap_540 (Trap_MediaReadfile) (implementation is correct, according to Oopho's snapshots).

Next: Implementation of trap_110 (Trap_AES) cases 0xFFF10000 and 0xFFF10001.

frogman

16th September 2008, 03:48

QUOTE Oopho2ei. I haven't expected such non-deterministic behavior.
It's a Sony!
Looking good boy's. :p

schluppo

16th September 2008, 05:27

I finished the Implementation of trap_110 (Trap_AES) cases 0xFFF10000 (AES_Encrypting) and 0xFFF10001 (AES_Decrypting). :rolleyes:

This means, that the VM is now running correctly without using any snapshots for 2,081,361 instructions containing 477 trap calls. It initializes the device, obtains AES-keys, loads ~500kb from 00001.svm and then decrypts these ~500kb.

Dumping memory after the 477th call gives a snapshot that is identical to Oopho's post_trap_mem_476.bin (he starts counting at 0) and the registers are identical to post_trap_reg_476.bin at this point. Also the PC is in sync. :)

http://uploaded.to/?id=gn1mmh
Sorry for the terrible source.

derbeDeus

16th September 2008, 09:36

well, you didn't really implemented the aes trap for player keys, did u? ;) hehe
and i don't think discovery will work with other titles either; you see... there's a key in the data for dev1/qid3

Oopho2ei

16th September 2008, 16:49

I finished the Implementation of trap_110 (Trap_AES) cases 0xFFF10000 (AES_Encrypting) and 0xFFF10001 (AES_Decrypting). :rolleyes:
Looks good so far. Please don't treat the output of some traps as constants if you are not sure they really are. Also what currently matters is that (1) your trap implementation works for all occurrences in the snapshots and (2) that you extensively comment your code. It's not important that you write perfect java code. As long as the code is sufficiently commented more experienced programmers can quickly fix all the mistakes you made.

Also could you please add your modified version to our repository? The codebase from Disabled is currently broken and hasn't been updated for a while.

bmnot

16th September 2008, 19:23

Some suggestions:
- for branches you better show the absolute address and the not the relative address
- could you automatically load the snapshots?
- add a data view window (maybe double the number of columns for the register view and cut its height by half to make space for a small data view window below)
- reset the watchdog timer with ever trap call to 0x7FFFFFFF
- display the parameters of the trap call directly before it is executed (e.g. by commenting the values in the stack window)
- support of data breakpoints
- color the registers red which have been changed since the last step/run
- maybe assign some hotkeys
okay... some good ideas :) . These are implemented in version 0.2.0 of the Debugger: http://uploaded.to/?id=f0sxrm

Some notes:
- I found it somewhat ugly to hard-code the snapshot-names for auto-loading, so there's a menu option "Get post-trap snapshots from directory...". The Debugger will collect the files with the correct size in this directory and sort them alphabetically. It will load the next snapshot in the list whenever a trap was executed.
- There are read and write data breakpoints (displayed red and green). These can be set by selecting an area in the memory view and pressing the appropriate toolbar button.
- The traps including descriptions are parsed from a file that looks like this:

000010: void TRAP_Yield(UINT32 sleepTime); ?",
000020: UINT32 TRAP_EventGet(UINT8 *evBuf, UINT32 len); ?",
000110: UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);",
000120: UINT32 TRAP_PrivateKey(UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord, UINT32 keyID); ?",
000130: UINT32 TRAP_Random(UINT8 *dst, UINT32 len);",
000140: UINT32 TRAP_Sha(UINT8 *dst, UINT8 *src, UINT32 len, UINT32 op); ?",
000210: UINT32 TRAP_AddWithCarry(UINT32 *dst, UINT32 *src, UINT32 len);",
000220: UINT8* TRAP_Memsearch(UINT8 *region, UINT32 regionLen, UINT8 *searchData, UINT32 searchDataLen); ?",
000230: void TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len);",
000310: void TRAP_Memmove(UINT8 *dst, UINT8 *src, UINT32 len);",
000320: 5 parameters -> ?",
000330: void TRAP_Memset(UINT8 *dst, UINT8 fillvalue, UINT32 len);",
000410: UINT32 TRAP-SlotAttach(UINT32 slot, UINT32 codeLen, UINT32 reqPriority); ?",
000420: UINT32 TRAP-SlotRead(UINT8 *dst, UINT32 slot);",
000430: UINT32 TRAP_SlotWrite(UINT8 *newContents);",
000510: UINT32 TRAP_MediaReadRequest(UINT32 offsetHigh, UINT32 offsetLow, UINT32 len); ?",
000520: UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);",
000530: 3 parameters -> ?",
000540: 5 parameters -> ?",
000550: 6 parameters -> ?",
000560: 4 parameters -> ?",
000570: 0 parameters ... nop/vendor specific?",
008010: UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);"

- The menu command "Load alternative trap descriptions..." allows loading modified such trap descriptions from a file. This can be useful if new information on traps is discovered - the Debugger will show updated, appropriate comments.
- Unfortunately, I haven't found time to look at the traps any further yet. This will be next. Hardening the VM with the 50000(!) :) snapshots is also interesting. Fortunately, schluppo already started on both :)
I integrated and slightly beautified his trap implementations (yesterday's version; hope I didn't break anything).
@schluppo It should be pretty straightforward to integrate your newest trap implementations.

The debugger is quite slow on inputting PC-traces. 4,000,000 PC values take 10min+.
Ok, this was clearly a flaw in the implementation - fixed that, too. Actually, this was only a minor detail:

File f = ...
/*before*/ for (int i = 0; i < f.length(); i++) { // hammering the file system
/*now */ for (int i = 0, l = (int)f.length(); i < l; i++) {
BTW, I could post more often. However, I am somewhat concerned about staying anonymous (although the activities should be perfectly legal here) - so I currently don't.

Oopho2ei

17th September 2008, 00:42

This is the output of aacskeys for this movie:MKBv: 4
Processing key: F190A1E8178D80643494394F8031D9C8
Encrypted C-value: 95247837F8A71FFDDBE8B0D3C2EE0575
Corresponding uv: 000000A0

Decrypted C-value: A174BE1DF3678FFCDBF8D43A7AF32B40
Media key: A174BE1DF3678FFCDBF8D43A7AF32BE0

Encrypted verification data: 810D5EE8B3011C4E8474E7702D12F55D
Decr verif data should be: 0123456789ABCDEF
Decrypted verification data: 0123456789ABCDEFD33493CD1DD12AC7

Drive FW info: 1.01

AGID: 00

Volume ID (AACSBYPASS): 1D2FC67462FDF986357A8F808FA1298B
Volume Unique Key: C5E868608ACD6700C2AC6A5E6AF533D9
Unit Key File Hash (DiscID): 6B63A40DD45F17A466B0C5CC35BF1500B4E0A9BA
Encrypted Unit Key 1: 6941CB89A6EEAB466AAB4B4F55DDA667

Decrypted Unit Key 1: 98E4D277F21AC464EFB0C0CA19FB50FD

I am posting this now because it's possible that some of these keys are used by the traps. It's unlikely that any of the important keys actually appear in memory though.
I have looked at the implementation of Trap_AES where the last parameter is a Key ID (in my particular case KeyID = 4). Like stated in the patent you have to first decrypt (aes-128-ecb) the encrypted aes key (pointed to by 4th parameter) using the player key and then use the resulting decrypted key to decrypt the data (second parameter) just like the last parameter were 0xFFF10001.
BUT the aes implementation doing the decryption of the key (first step) in my player has been transformed by some random mappings (different lookup tables basically) so the keys i could give you only work with the mapped aes version. Either i can figure out how to reverse the transformation or we have to use the mapped aes version for which i would provide a sample implementation in C. In the meantime you can play around with the keys above. Maybe it's one of them. :)

okay... some good ideas :) . These are implemented in version 0.2.0 of the Debugger: http://uploaded.to/?id=f0sxrm
Another great release. Thanks a lot! :)

tteich

17th September 2008, 10:07

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.
Something's wrong. The site appears to be offline.

schluppo

17th September 2008, 15:34

Here's a new edit of the debugger:
http://uploaded.to/?id=vj3a9w

I took bmnot's newest version and changed the following:

Put return value of traps to R1 after trap_execution.
If a snapshot is loaded, compare result of trap_execution with the snapshot. Output nice error message in console, if there are any differences in memory or registers.
If you select a snaphot-directory containing at least 500 or so snapshots, "Run" will now run till trap 490, without loading snapshots (in order to run a bit faster - since the first 490 traps are treated correctly and since loading and comparing snapshots takes some time). From trap 490 on, snapshots will be loaded (and trap execution results will be compared to them).
Treatment of traps. The following is an exhaustive list of TRAPs that I encountered yet:

0x110: TRAP_Aes();
Cases AES_Decrypt and AES_Encrypt are implemented correctly and generically.
Case AES_Decrypt_Media did not occur yet but is an easy case, provided that we have the mediakey.
The other cases use player keys which i don't have and are thus not handled in a generic way yet.
0x140: TRAP_Sha();
The only case that occured yet (SHA_BLOCK) is handled correctly and generically.
Other cases are simple and can quickly be added if needed.
0x230: TRAP_XorBlock();
This is handled correctly and generically.
0x520: TRAP_DeviceDiscovery();
This is handled correctly, but with constant results.
For qId=0x3, the result seems to depend on a key that I don't have, so this trap is not treated generically yet.
0x530: TRAP_???();
This trap is concerned with getting some data from the player.
It is treated correctly, but not in a really generic way yet.
0x540: TRAP_MediaReadFile();
This is handled correctly and generically.

Summarizing on the traps: As long as they do not use keys that I don't have, they are treated in a (hopefully) generic way.

Note, that this version of the debugger is still getting out of sync with PC-trace at about 3000 instructions after trap #494 (TRAP_SHA). But even if no PC-trace is used, the post_snapshot #495 contains more differences than are justified by the execution of trap #495 (TRAP_deviceDiscovery(1,1,...)). Since the traps are treated correctly according to the snapshots, I come to the conclusion that there is a bug in the debugger's basic VM. I don't have the knowledge to find and fix it though.

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.

Edit: This version of the debugger is also in the repository now.

Oopho2ei

17th September 2008, 16:58

Here's a new edit of the debugger:
Thank you! :)

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.
I have asked someone who i think got past this point. This will hopefully get fixed soon.

This is a list of interesting details i was told anonymously:
- Trap_DeviceDiscovery, Trap_PrivateKey and Trap_AES use player specific keys
- the virtual machine is event driven but there is no "Trap_EventGet" (events are passed in a different way)
- Trap_010 is Trap_Finish
- the trap vector table in the 00000.svm is sorted alphabetically by trap names (all the traps are called from this table ... you know which one)
- Trap_520(1,3, 0x4514, 0x1EFF84) outputs the volume id at 0x4514+0x18 so question id '3' == 'what's is the volume id of the current media?'
- ...

A big thanks for sharing this information with us. :)

Oopho2ei

17th September 2008, 17:44

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.
The person who got it running assumes your LB implementation is wrong and I was able to prove this :devil:.
Try the following: open the 00000.svm with a hex editor and overwrite the bytes at 0x1018 with the following sequence: "C8 00 10 04 88" This is cmd32 reading from 0x1004 (which is set to 0x88 by the last byte you replace). Now run the first command with the debugger. the result is stored in R0 and should be 0x88 but yours is 0xFFFFFF88. So somehow the byte is sign extended to integer.

Disabled

17th September 2008, 18:46

After a long time I'm reporting back. I stopped coding as the C99 and Java versions showed up, as I don't think there is a need for a third one.
Very good work of all of you contributing, including those anonymous ones!
On a side note, I fixed the "signed LB extension" bug in the svn. (Just remove the byte cast in L201.) I wasn't able to test it with the full run, as I don't have all the traces and files here, but it works on your test case and is now consistent with LW.
And I think it would be great to have more ppl hanging around the svn. At least bmnot should join it and I don't know who else is contributing or wants to contribute to the Java version or if there are more working on the C99 one they could of course use it too.

schluppo

17th September 2008, 19:01

Confirmed, removing the byte cast in L201 fixes this, now the debugger goes to trap #499 at instruction counter 6584939 with correct PC. :)

New traps Trap_Random (0x130), Trap_PrivateKey(0x120) and Trap-SlotRead(0x420) occur:

trap 494 is trap_520(1,1,0x1EFA94,0x1EF890->0x190); <works ok>

trap 495 is trap_140(0x1ef894,0x1efa94,7c,3); writes 0x7c bytes to 1ef894. <works ok>

trap 496 is trap_130(0x2348, 0x10); returns 0; puts 16 bytes to 0x2348: 2cb40ca4 65236776 0454806a 0eca1b92

trap 497 is trap_140(0x1ef924,0x1ef924,0x20,3); writes 0x20 bytes to 1ef924. <works ok>

trap 498 is trap_120(0, 0x5514, 0x2348, 0x10, 0); returns 0; writes 40 bytes to 5514:

4aa6ef81 dc4cf6b7 0237eaff e7d14a93
cfdc52e2 74811e16 2f770cfb ee11ce92
2a4e9fc4 beab67bd

trap 499 is trap_520(1, 1, 0x1ef794, 0x1ef794); <works ok>

PC goes out of sync between trap 499 and trap 500 due to nondeterministic behaviour.
Snaphot contents is ok though for at least the next 500 traps:

trap 500 is trap_110(0x5b430,0x5b430,?,?); returns 0; this case is not handled yet...

trap 501 is trap_520(1, 2, 0x1efcc8, 0x1efcc4); <works ok>

trap 502 is trap_520(1, 1, 0x1efe08, 0x1efcc4); <works ok>

trap 503 is trap_420(0x1efb14,0); returns 0. reads 256? bytes to 0xx1efb14.

trap 504 is trap_420(0x1efb14,1); returns 0. reads 256? bytes to 0xx1efb14.

trap 505 is trap_420(0x1efb14,2); returns 0; reads 256? bytes to 0xx1efb14.

the following 495 traps all read slots (there are 511 accessible slots, first 498 of them are read here).
Only the first slot contains data, the rest filled with 256? times 0x00.

Trap-Calls which are already handled correctly are indicated with "<works ok>" above...

Edit: New traps Trap_Slot-Attach (0x410), Trap_Slot-Write (0x430) appear...

Disabled

18th September 2008, 10:23

I took the adventure to download all the PC and Trap traces, about 1.5GB together. As I found that pretty inconvenient, I recompressed them using 7zip (http://www.7-zip.org/)and found they fit in less than 10MB! They still extract to ~20GB and are exactly the same as what Oopho2ei offered, but are much less on the download side. So here they are (http://www.zshare.net/download/18994624bff01fce/).
Under Windows I recommend extracting them to a compressed folder, then they will use just about 2GB of space and it will probably speed everything up, as the HD is the limiting factor for me. Linux users might use SquashFS or something similar.
As an extra and because I don't like uploaded.to that much I also uploaded the BDSVM folder here (http://www.zshare.net/download/1897968447300276/). *edit* This is the same as in post #130, just another server.

Oopho2ei

18th September 2008, 17:34

As I found that pretty inconvenient, I recompressed them using 7zip and found they fit in less than 10MB! They still extract to ~20GB and are exactly the same as what Oopho2ei offered, but are much less on the download side.
Awesome! I will make a package of the current movie ("The Day After Tomorrow") and of a "new" movie i have just bought ("I robot"). This time the instruction trace and the program counter trace will match the snapshots! :)

package structure:
/
AACS/
post_trap_snapshots/
BDSVM/
traces/
aacskeys_output.txt
title.txt
version.txt

Edit: i am currently compressing both packages so they should be available online in a few hours.

Edit:
Package 1 ("The Day After Tomorrow"): http://uploaded.to/?id=yodttu
Package 2 ("I robot"): http://uploaded.to/?id=mej65b

Oopho2ei

19th September 2008, 15:12

Upload complete (see posting #130). You should now have 3 independent traces (including the old/deprecated one).

ToDo:
- compare the two traces of "The Day After Tomorrow" to identify "random" data returned by the traps (like timestamps)
- compare one trace of "The Day After Tomorrow" with a trace of "I Robot" and look at the the result of the traps to identify disc/movie dependent data.

Good luck/have fun. I will continue to study Trap_AES as planned. :)

Edit: Trap_AES: the key with opOrKeyID=4 is a constant stored in the key section of the player. It's transformed and i need to reverse the transformation so you can use it with your unmodified AES implementations.

Edit: Trap_010 seems to update the Watchdog Timer. After the first call of Trap_010 in "The Day After Tomorrow" the timer is set to 0FA0h (or decimal: 4000) so program execution will break quickly afterwards. Very likely some changes are made to the guest system so don't panic if you run into problems after this trap. I will look into this soon.

Edit: After Trap_010 execution continues at *r28 (thanks to [anonymous])

Edit: I have compared two snapshots before and after the first Trap_020 call in "The Day After Tomorrow" and found no difference.

Edit: I made a snapshot before and after the first execution break (by the watchdog timer reaching 0) and found no differences. The watchdog timer was again set to 0FA0h (=4000) so this regular interruption is maybe to allow other player functions to be run on the processor of the player. But probably more likely this is a regular check for player events.

Oopho2ei

20th September 2008, 17:41

There seems to be a problem with the current post-trap snapshots after the first Trap_010 call. Please do not try to execute beyond Trap_010 and instead focus on the other traps which have been executed up to that point. I will notify you when this issue has been solved.

Edit: I've made new packages for irobot and dat which both contain an extra directory "post_break_snapshots" and an additional trace file "timer_trace.bin". Like with the post_trap_snapshots you have to implement a "break counter" which is incremented every time the watchdog timer (see timer_trace.bin) reaches zero (or becomes negative).

How to use this:
As usual you decrement the timer at the end of every instruction according to the table in posting #25. When this is done you check the value of the timer. Only if it is less or equal zero the program execution is interrupted and you load the snapshot post_break_???_[cnt].bin, increment the break counter [cnt] and then continue execution. As long as the timer is positive program execution continues normally and you don't have to worry about this. :)

Note: I now use %06d instead of %03d for the filenames.

The new packages will be available in a few hours.

Oopho2ei

21st September 2008, 10:57

The new packages are available now:

Package (v1.02) for "The Day After Tomorrow": http://uploaded.to/?id=g93e2g
Package (v1.02) for "I Robot": http://uploaded.to/?id=svpn7q

Oopho2ei

21st September 2008, 13:27

A quick note about the trap names: Some traps are mentioned in the pdf (http://securityevaluators.com/pdf/spdc_aacs_2005.pdf) which are not listed in the patent. For example TRAP_Finished (#010) and TRAP_DiscoveryRAM (possible #530). So maybe we can complete our list soon. :)

I keep on updating the list in posting #97: http://forum.doom9.org/showpost.php?p=1180641&postcount=97 Please use this as reference and do not make your own versions. If you see a mistake or maybe you want to add something please tell me. :)

schluppo

22nd September 2008, 14:55

Was busy the last few days, now I have time again...

Trap_Random (0x130) implemented, Trap_MediaReadFile (0x540) bug fixed. Trap_PrivateKey has different ordering of parameters than in the patent text:

UINT32 TRAP_PrivateKey(UINT32 controlWord, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 keyID);
vs
UINT32 TRAP_PrivateKey(UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord, UINT32 keyID);

Next: slot traps.

I am still using the old snapshot-series, will get the new snapshots soon and start looking at the traps which appear there.

Will upload new edition of the debugger soon...

Edit: TRAP-SlotRead, TRAP-SlotWrite, TRAP-SlotAttach implemented (in a rather sloppy way - need to check how these are treated for in other BD-discs).

Oopho2ei

22nd September 2008, 17:31

UINT32 TRAP_PrivateKey(UINT32 controlWord, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 keyID);
Can you please also tell us your observations which make you believe Trap#120 really is "TRAP_PrivateKey" with that signature? I am not saying you are wrong or anything. It's just hard to tell if you are (almost) sure or just guessing. :)

Btw. Please use the version number when you refer to a particular snapshot package. Note that we we have now packages from two different blue ray discs.

schluppo

22nd September 2008, 18:13

I was using the old version of "Day after Tomorrow" with the 5000 snapshots in 5 big packages. Apart from the AES-Traps which are using player keys (especially TRAP_AES(..,..,..,..,0)) and apart from TRAP_Random which should produce random numbers, the first 1600 of those snapshots are identical to the memory after execution of the coded treatment of those traps. Anyway, I'm extracting v1.02 of both movies now and will look at them soon in order to improve the treatment of traps.

Further, trap 0x120 could have any other name, I have no observation telling me for sure, that it's "Trap_PrivateKey". I was just using that name since it's in the trap-list at this position.

Also, a thing about trap 0x410: I'm not sure wether this really is TRAP-SlotAttach, since that should have three parameters, according to the patent. Until now, I found just one call of trap 0x410:

trap 1002 at ic 6732573 is trap_410(0x1f3,0xc) // slot-attach(499,12)?
returns 0x80000001

Notice, that I think, that 0x800000001 is the 'error' return status number.

Later, the program tries to read from this slot 499 (just receiving 256 times 0x00 and returning 0x0 in R1) and afterwards tries to write to slot 499 (returning 0x0 in R1 indicating successfull execution of the write-process?).

Maybe BD+ is using a more simple variant of this trap without security code / hash.

Edit: I compared the behaviour of the traps in the old snapshot-pack (the one with the 5 big archives) and the new v1.02 (both packs are for the movie "Day After Tomorrow"). Here is a list of traps with different behaviour in the first 1600 trap-calls:

TRAP_DeviceDiscovery(1,3,4514,1eff84) - qID #3: ~5 bytes difference due to timestamp
TRAP_120(0, 0x5514, 0x2348, 0x10, 0) (TRAP_PrivateKey?) - all 40 bytes of the result are different
TRAP-SlotRead(0x1efb14,0) - 32 bytes different content in slot 0

The other trap-calls behave in exactly the same way as before (apart from TRAP_Random of course).

Edit2: For "I Robot", there's the following problematic traps:

TRAP_DeviceDiscovery(1,3): ~5 bytes difference due to timestamp and 16 bytes difference since the Volume-ID of this BD is different.
I will implement this soon (we need a text file (or so) input anyway, containing all the keys needed to emulate the traps).
The AES-Trap behaves badly - as expected (we still need a way to emulate AES-decryption using player-keys).
TRAP_120(0, 0x54F0, 0x2348, 0x10, 0): all 40 bytes of the result are different.
At IC 2123976, trap_0x550(0x7F868,0xF,0,0x13FFE00,0x3DFCC8,0x7F850) writes 20 bytes to 0x7F850 and returns 0x0 in R1. Don't know what this trap could be.
Another call of this: trap_0x550(0x81FBC,0xF,0,0,0x3DFC0C,0x3DFC10) writes 20 bytes to 0x3DFC10 and returns 0x0 in R1.
One more call writing 20 bytes to the last parameter and returning 0x0 in R1: trap_0x550(0x3D5E8,0x16,5,0x27123800,0x3DFC00,0x3DFC04)
TRAP-SlotRead(xx,0): 68 bytes different content in slot 0.

Edit 3:
For trap 0x120 (TRAP_PrivateKey?), I found the following: It's second parameter is a pointer <pDst> where the result is going to be put. It's third parameter is a pointer <pSrc> to the input of this trap. It's fourth parameter <source_len> is the length of this input (this input is 0xFF-terminated). Further, this trap seems to behave non-deterministic.

I would guess, it is indeed TRAP_PrivateKey, just because the parameters are so close to what they should be. Imagine we have controlWord=0 and keyID=0 (indicating the case where this trap produces a signature of it's input, using a private key), then a call of this trap looks similar to the specification of the trap in the patent text:

trap_120(0, <pDst>, <pSrc>, <source_len>, 0)

vs.

TRAP_PrivateKey(UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord, UINT32 keyID);

Furthermore, the patent mentions, that the keyID is "normally" 0 for TRAP_PrivateKey, and in every call of trap_120 that I have seen so far, the first and last parameter were 0.

However, the patent also says, that for keyID=0, the result of TRAP_PrivateKey will be a "2048-bit RSA-Signature". I guess that means 256 bytes of result, but trap_120 just produces 40 bytes of result.

Oopho2ei

22nd September 2008, 23:33

The AES-Trap behaves badly - as expected (we still need a way to emulate AES-decryption using player-keys).
You can start to merge this into your code: http://uploaded.to/?id=w7qe3s
This sample program decrypts the given encrypted key using (a transformed version of) player key 4. The resulting decrypted key is transformed too so you need another obfuscated version of aes which i have not implemented yet. Together with this missing second half you will be able to use the player keys (to some extend) without really knowing them. These keys will probably be revoked quickly so it's not worth trying to remove all the different obfuscation layers just to find a "equivalent" key which can be used with a "normal" aes implementation.

Compile with:
gcc -o wbaes whitebox_aes_decrypt.c whitebox_aes_decrypt.h

Another thing: no trap is currently fully implemented because they all lack the initial parameter check. If a parameter combination is not allowed (eg. negative length, invalid address,...) the trap is supposed to refuse the execution (and maybe return 80000001h). But this isn't your fault because nobody has looked at this yet. :rolleyes:

schluppo

23rd September 2008, 02:56

Bug report for the basic debugger:

- Loading a big (>16mb) PC-trace file throws the following exception:

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(Unknown Source)
at java.nio.ByteBuffer.allocate(Unknown Source)

This prevents people from using a PC-trace beyond ~8,000,000 instructions. Even splitting the big file into smaller 16mb-files and trying to reload traces after every 8,000,000 instructions does not help and throws the same exception.

- The VM hangs up after Trap#2044 for I Robot, v1.02 / Trap#2029 for DAT, v1.02 (I manually compared PC to the respective PC-traces and it is for both titles in sync up to these traps). This happens due to PC getting value FE83AAFD+ and thus causing array out-of-bounds access. IC is at 38,756,448 / 39,061,664 by this time. :confused: I guess the program takes at this point a jump that confuses the debugger's VM to jump somewhere else than it should.

On another note, I added a log of the first 2000+ trap-calls for both titles. It should be self-explanatory and also contains all the differences between the 'implemented' traps and the snapshots: http://uploaded.to/?id=f8kiq8

LoRd_MuldeR

23rd September 2008, 05:31

The nasty "Java heap space" OutOfMemoryError happens all the time, when your Java app needs a certain amount of memory.
But it can be solved by giving the JVM more memory for the heap space. You can do it like this:

java.exe -Xms128m -Xmx768m ...

Oopho2ei

23rd September 2008, 08:12

This prevents people from using a PC-trace beyond ~8,000,000 instructions. Even splitting the big file into smaller 16mb-files and trying to reload traces after every 8,000,000 instructions does not help and throws the same exception.
Try to read only 1MB from the traces into memory and store it in a ring buffer like:
uint32 ringsize = 0x40000;

unit32 ringbuffer[ringsize];
uint32 i = 0; // entry pointer in trace file

fopen(...)

// initialize the buffer
fread(ringbuffer, sizeof(uint32), ringsize, trace_file);

while (1)
{
printf("current pc is: %08d\n", ringbuffer[i % ringsize]);

i++; // switch to next pc entry

// usually we could write and read at the same time (two threads)
// i assume only one thread so i refresh/write the entire buffer at once
if( i % ringsize == 0) // check if are back to our starting point (walked the ring all the way around). if so refresh the buffer
{
// read the next MB into the buffer
fread(ringbuffer, sizeof(uint32), ringsize, trace_file);
}
}
I wrote this in only a few minutes (so there might be a mistake) but i hope it helps. :)

Oopho2ei

23rd September 2008, 20:51

This sample program decrypts the given encrypted key using (a transformed version of) player key 4. The resulting decrypted key is transformed too so you need another obfuscated version of aes which i have not implemented yet.
Wrong. I couldn't find any obfuscation downstream. So the output is the decrypted key you can use in your default aes implementations:

Run the program given in posting #158. It outputs 4 dwords: "3817007A 0D652D45 0A23A6B0 95876211" so the decrypted key is:
7A 00 17 38 45 2D 65 0D B0 A6 23 0A 11 62 87 95
This is the data which has to be decrypted with Trap_AES in "The Day After Tomorrow" using player key 4 (it's Trap call number 6):
21 BB 37 02 E1 CF F6 EE 92 82 3F B5 A0 A1 35 00
3D 97 95 F0 AB F6 41 86 9D 51 07 D7 E9 80 7F BC
B8 94 BB 0D 7F 72 34 C4 16 4C 6B 40 BC 8F 92 2C
Now i can again simply put this into a file "enc_data.bin" (appended some zeros...) and execute:
openssl aes-128-ecb -d -K 7A001738452D650DB0A6230A11628795 -iv 00000000000000000000000000000000 -in enc_data.bin -out decr_data.bin

The result is in decr_data.bin:d2 8a e3 fd 2e 66 63 55 b7 64 01 6e 4a da 41 2c
dc 7d fb e9 46 13 4b 76 5c 83 27 1c c6 6b 38 6a
00 00 00 00 00 00 00 00 00 00 00 01 00 00 44 22
Which matches the snapshot. :helpful:

Oopho2ei

23rd September 2008, 22:42

Those are the keys which can be used by Trap_AES (you already knew key 4 from the source code):
Key 0: B0 BF 21 C3 5C 19 A2 A1 E3 93 71 5F 4A BA F7 BD
Key 1: 90 13 DA 1D 3D F3 34 F9 D5 F1 7C 31 17 25 40 AC
Key 2: 3A CA 33 02 80 3E 29 F8 BD C0 21 24 B3 09 40 BC
Key 3: 3A E5 B0 C1 8D 32 04 85 32 CC 7C 4C BC 87 37 22
Key 4: DA 15 85 51 1B DF 6F EC D2 30 A5 4E 85 B4 57 2E
Key 5: 2E AD EA A1 96 D1 BC 7D F0 4B E0 51 DB D0 FA C0
Key 6: D5 A7 3B 9E 18 15 C4 D1 0E FD D0 BC 21 C0 86 D3
They only work with "my" obfuscated aes implementation given in posting #158 so it's perfectly correct to "hard code" them in your source. If someone feels bored and removes some obfuscation from the algorithm this will likely also change the keys. The two movies i provide snapshots for ("The Day After Tomorrow" and "I robot") only seem to use Key 4 and Key 5 together with Trap_AES. I think Trap_DeviceDiscovery accessed Key 0 too but i have to check this again... not sure. :)

This should be all you need to implement Trap_AES for the remaining parameters now. I will probably look at parameter checking next.

Good luck with your implementations! :p

Edit: the obfuscation scheme has been broken. Below are the keys you can use with your default/plain aes implementations:
key00 = 6C1AC6780E996094F04AEA2B12CCC382
key01 = B2E16A58560F8AF59E47881D037B5CDF
key02 = AD08B3F2571247488B1AB975137B707B
key03 = 6E8B9CF22A3F4B45E347B5FA8D0CFE74
key04 = FEBE6C124354A6D3E19E491A816CCD4D
key05 = 0ED1D4E6D287A85EFEDB32386FC1A913
key06 = 3100DE1D7EFF6CD013EB84C67CBDB9E9

schluppo

24th September 2008, 01:04

Debugger problems solved:

- Using "java.exe -Xms128m -Xmx768m -jar Debugger.jar" allows for loading the 150mb of pc-trace, so this problem is solved (for now without using a ringbuffer or similar). Thanks for the hint.
- The INSTF instruction was not correctly implemented. I corrected this and now the debugger goes without problems past trap #2044 (I Robot)/#2029 (DAT).

On another note: Great work on the AES-trap!

Edit:

Yet another bug in the basic VM:

for DAT v1.02, at IC=37079238, PC=0x357680, instruction 0xB00DFA1C is executed. This should be "JMP 0x370A0" but is interpreted as "JMP 0x4370A0". The first is a jump backwards 0x3205E0 bytes, the second is a jump forwards 0xDFA20. 0x3205E0 is (more or less) the 2's complement of 0xDFA20. So... the current debugger's VM doesn't handle long backward jumps right.

I am no bitshifter, so could someone help out?

Current code:

long i = IF ^ unsigned(mem32.get(pc / 4));
...
int JimmS = ((int)(i << 6)) >> 6; // Jump constant signed
...
case 0x2C: pc += JimmS; break; // J

Oopho2ei

24th September 2008, 08:20

This should be "JMP 0x370A0" but is interpreted as "JMP 0x4370A0".
You forgot the "AND 0x3FFFFC" (see posting #24). You have to do this for every address otherwise content code can break out of your sandbox environment and potentially execute arbitrary code on your computer. :devil:

schluppo

24th September 2008, 19:00

Ok, I got this fixed along with a few other bugs in the debugger's basic VM. It's staying in sync with PC till trap-call #2676 now (all the following is for DAT v1.02 - but I Robot v1.02 behaves quite similar).

The VM seems to self-check itself starting around trap-call #2000. It does some INSTF-tampering, but changes INSTF back to 0 and keeps it at 0 after a few hundred instructions. Further, it does very long / backward jumps or execute non-jump instructions at 0x003FFFFC afterwards resuming execution at 0x00000000. Around trap-call #2100, the bad parameter trap-calls are starting, such as reading from slot 0xFFFFFFFF (which does not exist) or reading (writing) from (to) memory outside of range for several traps. I fixed some of those, but some others remain to be treated.

Trap-call #2676 is finally the first call of trap_0x10(0);

And soon, I will upload that promised new debugger-edit and also logs of the first ~2700 trap-calls for both movies.

Edit: new version of the debugger: http://uploaded.to/?id=z3gq5g, logs: http://uploaded.to/?id=71kv7r

Oopho2ei

24th September 2008, 23:33

I have pushed some documentation into our repository: http://svn.assembla.com/svn/bdplus

To checkout the repository use:svn co http://svn.assembla.com/svn/bdplus

schluppo

25th September 2008, 00:54

Things are getting more interesting, I modified the debugger to get past the first few calls of trap_0x10 and the consequent breaks using the break_snapshots.
Traps 0x20 (GetEvent?), 0x510 (MediaReadRequest?), TRAP_Memset (?), TRAP_AddWithCarry (?), TRAP_MemMove (?) appear.

frogman

25th September 2008, 17:27

I'll just say: due to certain properties of BD+, once you're past a certain point, you can handle it pretty much without reversing - BD+ itself then helps you out - on any player ;)

I am wondering how true this statement really is.... Oopho2ei and schluppo you guy's must be pretty close to solveing the open source solution.

Dam good work.

Slysoft, Is still the best investment out there!:thanks:

Oopho2ei

25th September 2008, 17:54

I am wondering how true this statement really is.... Oopho2ei and schluppo you guy's must be pretty close to solveing the open source solution.
I think he was referring to all the self checks done before the content code actually does something useful. We have so far passed the memory management and instruction checks. Now a lot of trap checking lies ahead so the player interface is tested extensively. Once we have passed all those checks without the help of the snapshots (guided execution) we should be pretty close. Currently it's still a lot of work.
Btw it's not only me and schluppo. There are other people hiding behind my back which have so far helped me a lot. Big :thanks:

schluppo

25th September 2008, 18:26

I added parameter checking for the following traps to the debugger:

0x140 (Trap_Sha)
0x210 (Trap_AddWithCarry)
0x220 (Trap_MemSearch?)
0x230 (Trap_XorBlock)
0x310 (Trap_MemMove)
0x320

Edit:

new debugger: http://uploaded.to/?id=mq66ms

- Allows to load timer-trace-file (watchdog-trace). shows actual and expected watchdog-counter value. allows to manually set wd.
- Added option to set post_break_snapshot directory. If watchdog reaches 0, automatically load post_break_snapshot and compare registers + memory (same as with traps).
- Added break counter which is increased everytime, when post_break_snapshot needs to be loaded.
- Manually set trap-counter, instruction counter and break counter.

Here's an example of how to use this. I will load the machine to the state after trap-call #2676 in DAT v1.02.

So, put DAT v1.02 00000.svm, 00001.svm and 00002.svm inside the dir where Debugger.jar is. Start the debugger like this: java -Xms128m -Xmx768m -jar Debugger.jar. Load DAT v1.02 00000.svm. Load DAT v1.02 pc_trace.bin, Load DAT v1.02 timer_trace.bin. Load Memory: DAT v1.02 post_trap_mem_002675.bin, Load Registers: DAT v1.02 post_trap_reg_002675.bin. Set IC to 0x2382BDA (37235674), set PC to 0x1AC00, set TC to 0xA74 (2676). Now everything is set to the state past trap-call #2676 in DAT v1.02.

- Note: This debugger-edit also contains a non-working port of Oophoo's modified whitebox aes to java (functions XAES_SetKey(), XAES_Decrypt(), XAES_Block() and so on). I am no AES-expert, so I would appreciate any help in getting these non-working functions to work.

Edit 2: Player key AES-decryption added to the debugger - works fine now :devil: (thanks to Oopho). Lots of small bugs fixed. Will upload new version of the debugger soon.

Edit 3: Here it is: http://uploaded.to/?id=xbayxh. You can go to trap-call #3943 (DAT v1.02) / #3968 (I Robot v1.02) with PC/WD in sync. Trap-call logs up to this point: http://uploaded.to/?id=oe3azr

Oopho2ei

26th September 2008, 15:09

A few things which really need to be improved/changed:
private int trapXorBlock(int pDst, int pSrc, long len) {
if ( (pDst & 0x3FFFFC) >= MEMSIZE ) return 0x80000001;
...
if (((pDst & 0x3FFFFC) & 0x03) != 0 ) return 0x80000001;
...
If only one of the trap parameters is invalid (out of memory bounds) you don't execute the trap and return STATUS_ERROR (see status codes (http://svn.assembla.com/svn/bdplus/trunk/doc/misc/status_codes.txt)). The "& 0x3FFFFC" fools my parameter checking into 'thinking' that "pDst & 0x3FFFFC == pDst" so don't use it. You can use the mask when executing the trap just to be sure the content code can't break out of your sandbox in case the parameter checking is incomplete. If you are sure the parameter check covers all bad cases you should omit the mask from the entire trap.

The dword,word and byte masks (0x3FFFFC, 0x3FFFFE, 0x3FFFFF) can be derived from the memory size:
uint32 memsize = 0x400000;
uint32 MASK_DW = (memsize - 1) & 0xFFFFFFFC;
uint32 MASK_W = (memsize - 1) & 0xFFFFFFFE;
uint32 MASK_B = (memsize - 1) & 0xFFFFFFFF;
You should replace all occurrences of 0x3FFFFC etc. with the appropriate constant.

Finally please remove the case statement from Trap_AES and use something like:
if (opOrKeyID < 0xFFF10000)
{
If (opOrKeyID >= player_keys_count) BUG("TRAP_0110: unavailable key accessed") // some self checks (player_keys_count is currently 7)
...
... player_key[opOrKeyID] ...
...
return ... ;
}

if (opOrKeyID == 0xFFF10000)
{
...
return ... ;
}

if (opOrKeyID == 0xFFF10001)
{
...
return ... ;
}

// if the below statement is executed there is a bug in the parameter checking
BUG("TRAP_0110: invalid opOrKeyID");
After the parameter check (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/param_check/0110.txt) passed opOrKeyID can only be 0,1,2,3,4,5,6,0xFFF10000 or 0xFFF10001. It's not a good programming style to copy/paste the same code 7 times in a row. :rolleyes:
If one of the BUG("...") statements is executed the parameter checking is wrong/incomplete.

Thanks for your new release. :)

Edit: and please only use spaces to format your code (no tabs). My obfuscated aes implementation looks really messed up (another layer of obfuscation added :D)...

Edit: added some more details to my suggestions

schluppo

26th September 2008, 17:24

I integrated your requests.

Furthermore, guided execution reached the end of both snap-shot packages (with PC/WD in sync). So far, the implementations of TRAP_XorBlock and TRAP_Aes behave fine for every call. Will add / fix parameter checking of other traps now.

Note that there remain several problematic traps which are not treated yet, such as 0x530, 0x550, 0x120.

Edit: Fixed TRAP_Random, TRAP_MemSet and TRAP_MemMove. They behave nice now for all calls (parameter checking ok too).

Edit2: TRAP_AddWithCarry implemented and it behaves nicely for all calls (parameter checking ok). There's a small bug in the parameter-checking from the repository though:

if ( dst + 4*len > src & dst <= src + 4*len & dst != src) return 0x80000001; (correct)
vs.
if ( dst + 4*len > src & dst >= src + 4*len & dst != src) return 0x80000001; (wrong)

TRAP_MediaReadFile and TRAP_DeviceDiscovery fixed and they behave nicely for all calls (TRAP_DeviceDiscovery results seem to be constants except for params device=1 and qId=3).

Edit 3: Trap 0x220 was in fact not TRAP_MemSearch but TRAP_MultiplyWithRipple(int pDst, int pSrc, int len, int multiplicator). It's implemented now, parameter check is ok and all calls of this trap behave nicely.

Oopho2ei

26th September 2008, 20:48

Thanks.

I have finished the documentation of the parameter check for trap groups 01XX, 02XX and 03XX: http://svn.assembla.com/svn/bdplus/trunk/doc/traps/param_check/

Group 04XX and 05XX will follow soon. :)

Btw trap #0120 is almost certainly UINT32 TRAP_PrivateKey(UINT32 keyID, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord);

Oopho2ei

27th September 2008, 13:46

Edit2: TRAP_AddWithCarry implemented and it behaves nicely for all calls (parameter checking ok). There's a small bug in the parameter-checking from the repository though:

if ( dst + 4*len > src & dst <= src + 4*len & dst != src) return 0x80000001; (correct)
vs.
if ( dst + 4*len > src & dst >= src + 4*len & dst != src) return 0x80000001; (wrong)

Almost. It's "dst > src + 4*len". I forgot to negate the check. Thanks!

TRAP_DeviceDiscovery results seem to be constants except for params device=1 and qId=3).
For dev=1 and qID=3 it outputs the volume id. Open the "aacskeys_output.txt" in your package and have a look. :)

Edit 3: Trap 0x220 was in fact not TRAP_MemSearch but TRAP_MultiplyWithRipple(int pDst, int pSrc, int len, int multiplicator). It's implemented now, parameter check is ok and all calls of this trap behave nicely.
Great. I have updated everything and added parameter checking for traps #0510 and #0520 to the repository. (revision 35 now)

Edit: can you make a list of possible status return codes for every call? When i write "return 0x80000001" it doesn't necessarily mean that 0x80000001 is written into r01.

Edit: i am done documenting the parameter checks. The next step is TRAP_PrivateKey() :)

schluppo

27th September 2008, 22:04

I added the other cases (SHA_init, SHA_finish, SHA_update) and parameter checking to TRAP_Sha. Intermediate results do not match yet, but results after SHA_finish match (patent says, intermediate results are undefined but "can be used to store data"). Apart from intermediate results, TRAP_Sha is fine for every call.

My guess for trap 0x320 is TRAP_MemSearch, will look into that soon.

Edit: Here's a new version (including parameter checks for all traps, feel free to bugfix or edit it yourself :)):
debugger: http://uploaded.to/?id=jtslin, logs: http://uploaded.to/?id=ave7c7

Traps 0x120 and 0x530 seem to be most important to me now (since they require player-data to be emulated). The fixing of the problems with other traps (0x320, 0x140, 0x4*0, 0x520) just takes some more time to implement.

Oopho2ei

28th September 2008, 11:57

1. You should work more with our repository so it contains always the latest version. Only upload source and documentation there and no files which can be derived from the source like binaries and logfiles.
I have updated the source in the repository now. You can still upload the binaries to "uploaded.to" for a daily/weekly snapshot.

2. In my parameter checking there is no bit masking and still you use this "MASK_UNS" which masks the most significant bit. All the checks should work perfectly as they are. If they don't please report the problem and we will see how to fix it.

3. You should store the player keys in an array of arrays so you can access them like this OBF_PLAYER_KEY[opOrKeyID] together with a range check.

if (((int)opOrKeyID) == 0x00000000){
return(obfAES(OBF_PLAYER_KEY0,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000001){
return(obfAES(OBF_PLAYER_KEY1,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000002){
return(obfAES(OBF_PLAYER_KEY2,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000003){
return(obfAES(OBF_PLAYER_KEY3,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000004){
return(obfAES(OBF_PLAYER_KEY4,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000005){
return(obfAES(OBF_PLAYER_KEY5,pDst,pSrc,len,pKey));

}

if (((int)opOrKeyID) == 0x00000006){
return(obfAES(OBF_PLAYER_KEY6,pDst,pSrc,len,pKey));

}

It replaces all this by a single statement. :rolleyes:

4. I would like to see a separate file for each of those:
- instruction processing
- decoding of cmd39, parameter fetching, parameter checking, return code interpretation
- trap implementations
- obfuscated aes implementation and player keys

So the code looks well structured and we don't squeeze everything into BDVM.java which will grow larger over time.

Traps 0x120 and 0x530 seem to be most important to me now (since they require player-data to be emulated).
I am working on TRAP_PrivateKey() (#0120)

Oopho2ei

28th September 2008, 16:20

For the first call of UINT32 TRAP_PrivateKey(UINT32 keyID, UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord) in dat with parameters:

keyID = 00000000
dst = 00005514
src = 00002348
srcLen = 00000010
controlWord = 00000000
The 16 bytes from src are placed after a 16 byte header consisting of "BDSVM_PK" and two dwords. The first dword is probably the control word and the second dword is probably the length. The result looks like this:
42 44 53 56 4d 5f 50 4b 00 00 00 00 00 00 00 10 <-- header "BDSVM_PK"....
35 52 B5 2A F2 7D BA 60 97 93 9C 66 5F 5C 84 F3 <-- 16 byte from src
The result is then hashed with SHA-1. Again i put these 32 bytes in a file and called "sha1sum -b sha_in.bin". The output is (spaces added):
b99b5c49 af55cb7e 9b1e7d90 6846d720 2b9c2908 *sha_in.bin
Which is correct. Now this is supposed to be signed with rsa. Let's see ... :)

Edit: I have modified the example so it matches the DAT v1.02 snapshots.

schluppo

28th September 2008, 17:57

1. Will do.

2. There were a few bugs in your parameter checking (usually mixup of '<' and '>' or '>=' instead of '>' etc.). I fixed those that came to my eyes. Some are still left - see the logs. And concerning MASK_UNS: Java interprets integers as signed numbers. So when doing this:

int bla = 0xFFFFFFFF;
if (bla>0x10) return 0x80000001;
return 0;

it will return 0 (since 0xFFFFFFFF is interpreted as a negative number), but for parameter checks we would want it to return 0x80000001 in this case. Hence I masked the first bit to avoid this trouble. Since it works as it is and my time is limited, feel free to work around this problem in any other way (by using longs to hold the parameters?).

3. Done

4. Will do so, if I find the time.

On another note: What you said so far about TRAP_PrivateKey can easily be coded :)

Oopho2ei

28th September 2008, 19:50

2. There were a few bugs in your parameter checking (usually mixup of '<' and '>' or '>=' instead of '>' etc.). I fixed those that came to my eyes. Some are still left - see the logs.
Please let me see what you found so i can verify it and update my documentation. :)

And concerning MASK_UNS: Java interprets integers as signed numbers.
Argh! Then we have to use long. We can't ignore the MSB in the parameter check. I did a quick search and a lot of people are furious about the lack of unsigned types (except for 'char') in java. :(

On another note: What you said so far about TRAP_PrivateKey can easily be coded :)
I have hit heavy obfuscation. It almost looks like the rsa procedure is executed on another virtual machine with a lot of garbage code. This will take some time...

schluppo

28th September 2008, 21:02

Some info about trap return values: So far, I have only observed the return values 0x0 (STATUS_OK), 0x80000001 (EXECUTION_ERROR?) and 0x80000002 (BAD_PARAMETERS?). I never saw any trap returning its value in another register than in R1. Only trap 0x110 is changing PC and WD, the other traps don't change any machine data besides memory and R1. And there are some traps which don't return anything (trap 0x10, 0x230). This last thing still needs to be handled correctly at the end of trap execution (easy to do).
And here's some information about TRAP_PrivateKey, taken from the patent (in case you didn't read it yet :)): " For the player's main RSA private key (keyID=0), this operation produces a 2048-bit RSA signature of a SHA-1 hash. The hash is computed as follows: (a) hashing the value of srcLen, encoded as 4 bytes (MSB first); (b) hashing the contents of the user-specified buffer (i.e., srcLen bytes at src); (c) hashing the value of controlWord, encoded as 4 bytes (MSB first); (d) If controlWord bit 31 (the MSB) is set, hashing the value of the media ID; (e) If controlWord bit 30 is set, setting the destination pointer to PC+4, overriding dst; (f) if controlWord bit 29 is set, hashing the (control word mod 2.sup.16) code bytes beginning with the current program counter; then (g) if controlWord bit 28 is set, incorporating the current value of PC in the hash. The SHA-1 hash result is then padded by prepending "0x001.parallel.0x01.parallel.0xFF (repeated 233 times).parallel.00" to the SHA-1 hash. The padded value is then raised to the player's secret exponent, modulo the player's public modulus. "

Oopho2ei

29th September 2008, 00:19

Thanks.

TRAP_PrivateKey: I was able to trace both the obfuscated key (~ 96 byte) and the hash result to the heavily obfuscated code (virtualized, lot's of garbage,...). Not sure if i can do it. You would probably kill me if i convert this mess to c code. :D

Looks like i have to record a instruction trace and then somehow analyze it automatically. I need to think about it.

Edit: Any idea of how the result of TRAP_PrivateKey is verified? It's obviously a 320-bit signature which is probably not created by the RSA algorithm. The footer of every section of in the 0000?.svm files is 40 bytes long too. Note that the signatures used in the AACS authentication process are 40 bytes long as well. The way the result of of TRAP_PrivateKey is verified would give us a vital clue of what algorithm created the signature.

Edit: When the hash is all zero the resulting signature is also all zero. If i overwrite the key with zeros i get stack overflows (wtf?)

schluppo

29th September 2008, 21:32

Actually, until trap-call ~5000, this trap is called just once with real parameters (for both movies). All other calls of this trap have bogus parameters and return 0x80000001.

And for this one real call, the program directly calls DeviceDiscovery(1,3) afterwards. The 40 bytes which are written by this call of TRAP_PrivateKey are (according to 'data read breakpoint' of the debugger) not accessed in the next 20 million instructions. Furthermore, before overwriting the register which is holding the adress of these 40 bytes, the progam writes this same adress to two places in memory which are also not accessed in the next 20 million instructions. So to me it seems as if this result is not verified at all (or maybe way later in the BD+ initialization).

I will try and check a bit more, whether I can find any code accessing the 40 bytes.

Oopho2ei

29th September 2008, 21:59

Just like i expected: the unknown algorithm produces different results for the same hash (and the same key). It furthermore always produces a signature consisting entirely of zeros (wtf?) when the input hash is all zero. So now it is even more likely that it's some elliptic-curve signature algorithm like the one used in AACS. I now need to find the third (random) parameter... :devil:

The 40 bytes which are written by this call of TRAP_PrivateKey are (according to 'data read breakpoint' of the debugger) not accessed in the next 20 million instructions.
Do the data read breakpoints work reliable yet? Maybe they are cleared when a new snapshot is loaded? Or maybe the result (40 bytes) is not accessed by the content code (instructions) and instead another trap call reads them? There must be a check somewhere... :confused:

Oopho2ei

30th September 2008, 08:00

I was told the first byte from Trap_PrivateKey(...) is accessed by instruction "C8440000h" at address "091A98h" (this is instruction number 6585958). So there must be a problem with the data breakpoints of the debugger.

schluppo

30th September 2008, 08:27

Hm, if I find some time, I will look into the breakpoint-system of the debugger.

On another note, some info on trap 0x550: It is not called in "DAT" v1.02, but it is called three times in "I Robot" v1.02.

It takes as first parameter a pointer to these strings:
1) "BDSVM/00002.svm"
2) "AACS/MKB_RO.inf"
3) "BDMV/STREAM/000012.mts"
The second parameter is the length of this string.

Parameter three holds 0x00, 0x00 and 0x05. Fourth parameter is either 0x00 or a big value (>0x3FFFFFF) - this is not a pointer, but four bytes of data.

The fifth parameter is a pointer to 0x200. The sixth parameter is the destination pointer (this trap seems to always write 20 bytes).

I assume this trap is computing some kind of hash of the file. Could be CRC or SHA, my bet is on SHA since that outputs 20 bytes anyway. The values in parameters three, four and five should be additional inputs to this hashing algorithm.

Oopho2ei

30th September 2008, 13:40

According to your description Trap#0550 would be something like:

UINT32 TRAP_MediaHashFile(UINT8 *FileName, UINT32 FileNameLen, UINT32 Unknown, UINT32 Unknown, UINT32 *len, UINT8 *dst);

When you look at the parameter checking (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/param_check/0550.txt) you will see that it doesn't always produce 20 bytes. Instead it writes 20 * ( mem[len] / 0x200 ) bytes. Also the filename/path length may not exceed 1024 (!) bytes. :eek:
One of the two unknown parameters could be a file offset.

schluppo: could you please fix the data breakpoint problem? I need the value of n for testing (see ECDSA (http://en.wikipedia.org/wiki/Elliptic_Curve_DSA)). :)

Edit: Someone gave me a hint how to fix the problem with the data breakpoints. I have pushed the fix in the repository. :thanks:

Oopho2ei

30th September 2008, 23:12

I currently don't have the curve parameters (and therefor can't present a real example) but i don't want to withhold anything. The signature is created with ECDSA and consists of two 20 byte long numbers: r followed by s. How the signing process works is explained here (http://en.wikipedia.org/wiki/Elliptic_Curve_DSA).

You can forge signatures without the curve parameters simply by using a fixed point on the curve all the time which means your random number k and r (first 20 bytes) are constants.

So these are all the constants you need:
r = 7c7d84d310c42436daff10af116f80448dc3f88b
k = 7AA9A10C3E90AA8FDAF34DD28F665D3C609CACD7
d_A = 735E0356EA6356E099AD83152E2D81EA1164A866 (keyID == 0)
n = 96609d9e935e52c683dafdc49216143f9a24373d

That's the only variable - our message hash we have to create a valid signature for (see posting #178 about how this is calculated):
e = 1111111111111111111111111111111111111111

so now you calculate s: s = 1/k * (e + r * d_A) mod n

The result is: s = 532b4944f80565075ea9b8894e61c692ed71f3c8

Edit: this is arithmetic over a finite field with the numbers 0,1,2,...,n-1. Any other number (negative, fraction,...) doesn't exist.

Edit: you need the extended euclidean algorithm to calculate 1/k. The result is "1/k = 210974BC85725B5FB0E04FA473FD988001F89088h". So simply multiply with that value.

Edit: There were problems with the previous values so this is a new example and a new private key. It has been tested and should work fine.

Edit: If you the signature verification fails then TRAP_DeviceDiscovery will be called instead of TRAP_Aes after TRAP_PrivateKey (first call of TRAP_PrivateKey in DAT v1.02)

Edit: That's the second private key:
d_A = 87E22D06A699EE311CD73F19B91E67414C44F4F5 (keyID == 1)
It's the same curve (same parameters) so you can use the same (k,r) pair.

Edit: We have found the remaining curve parameters: q = 96609D9E935E52C683DBFC3A7D783EA942BDE8CB
a = 96609D9E935E52C683DBFC3A7D783EA942BDE8C8 = -3 (mod q)
b = 3E567D8DEC27873BCF86F5FBB595DB288C62C721
x_G = 05FC5B0B2360AC50A76E1511BC5C9AF67A004D0D
y_G = 09B0D43F319B09A5B679CCF264E1ABA4D56594EA

The corresponding public key for keyID == 0 (can be calculated from above values):
x_Q = 31dbb17fea9d7d0eb8bfbb6b8d29ac938c13b599
y_Q = 2465dd38f4214edee4da4e7f62b77a41ce962757

The corresponding public key for keyID == 1 (can be calculated from above values):
x_Q = 57305D64D1013FFFA4D72355B4C70B27A815A343
y_Q = 05E22B42F88A5C1AC5E70DF466353BDB0A83228A

Great job everyone! :)

Oopho2ei

1st October 2008, 08:01

Regarding Trap#0550 i was told that the fourth parameter is the file offset and that this trap really calculates the SHA-1 hash.
UINT32 TRAP_MediaHashFile(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst);

It works fine unless the third parameter (still unknown) is not zero but 5. We don't know what this means yet. Maybe only every fifth block of the file is hashed. Or maybe a block larger than 512 bytes (5*512 bytes) is hashed and results in a 20 byte SHA-1 hash. This still has to be figured out... :)

Edit: The file offset is 64 bit (FileOffsetHigh << 32 | FileOffsetLow). A 32 bit address is too small for those large files (00001.m2ts is > 20GB).

Edit: TRAP_MediaHashFile is fully analyzed now and i have updated the documentation in svn (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/)

Edit: We need proper names for Trap#0540 and Trap#0550. In alphabetical order name(Trap#0540) has to stand before name(Trap#0550) which is currently not true:
name(Trap#0540) = TRAP_MediaReadFile <-- maybe change in TRAP_MediaFileRead ?
name(Trap#0550) = TRAP_MediaHashFile <-- or maybe change this in TRAP_MediaSHA1HashFile
Suggestions?

Edit: The naming issue is solved. The "official" name for Trap#0550 is TRAP_MediaSHAFileHash

schluppo

2nd October 2008, 04:49

Great work on TRAP_PrivateKey! :)

Trap 0x550 is implemented now. Works fine (for the two files that I have :rolleyes:).

I named it 'trapMediaHashFile' in my code but do not care about the name actually. Just change it to whatever you think is appropriate.

trap 0x320 is trapMemSearch, will implement it soon.

is it possible to code trapPrivateKey yet or should I wait until you posted some more examples / parameters?

Only big thing remaining now is trap 0x530 aka DiscoveryRAM. This is called around 500 times, taking memory from lots of different areas of the player's RAM. Maybe we need a memory dump and let the trap-emulation read from that dump-file or something like that. :confused: I checked a few example trap-calls with identical parameters and did not see differing results. Maybe this trap is just asking for areas of the RAM which are constant throughout execution?

Oopho2ei

2nd October 2008, 12:30

trap 0x320 is trapMemSearch, will implement it soon.
Are the parameters identical to the TRAP_MemSearch() in the patent?

is it possible to code trapPrivateKey yet or should I wait until you posted some more examples / parameters?
You can start to implement it now. I am currently looking at events. The area 0x000-0xFFF seems to be used for passing events after the content code signals it's readiness to the player with Trap_Finished().

Only big thing remaining now is trap 0x530 aka DiscoveryRAM. This is called around 500 times, taking memory from lots of different areas of the player's RAM. Maybe we need a memory dump and let the trap-emulation read from that dump-file or something like that. :confused: I checked a few example trap-calls with identical parameters and did not see differing results. Maybe this trap is just asking for areas of the RAM which are constant throughout execution?
If you need to know where this data comes from i will look at this trap.

schluppo

2nd October 2008, 14:02

It's like this:
trapMemSearch(UINT8 *Region, UINT32 RegionLen, UINT8 *SearchData, UINT32 SearchDataLen, UINT8 *Dst)
I will be gone for a few days now. But I will try to implement 0x320 and fix 0x140 and 0x520 in the meantime.

Oopho2ei

2nd October 2008, 16:03

It's like this:
trapMemSearch(UINT8 *Region, UINT32 RegionLen, UINT8 *SearchData, UINT32 SearchDataLen, UINT8 *Dst)
Thanks. I assume at "dst" a pointer to the first match is stored? Also it can't be "UINT8 *Dst" because of:if ( Dst & 0x03 != 0 ) return 0x80000001

One of our anonymous contributers has suggest to use a optimized format for our snapshots. Instead of always storing the full snapshot only the differences to the previous snapshot are saved. This saves a large amount of disc space and probably also speeds up execution because the harddrive is currently the bottleneck.
The implementation would look like this: you load the first snapshot and compare as usual and continue execution but keep the last snapshot in memory. For the next snapshot you use the previous snapshot (saved in memory) and the difference to the previous snapshot (which is stored on disc now).
A sample implementation of how to handle the "differential snapshots" is given here (http://svn.assembla.com/svn/bdplus/trunk/diffarchive/)

I have updated the trap documentation of TRAP_Memsearch and made some minor changes to wbaes.

Also what traps actually return a value (write to r01)? According to the documentation (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/overview.txt) most of them return "UINT32". I was only guessing. :rolleyes: Has anyone verified that?

Edit: All traps for which the parameters are checked return a value. I have corrected the documentation and also renamed Trap#0550. The "official" name is now TRAP_MediaSHAFileHash. Thanks again to our anonymous contributers :thanks:

Oopho2ei

2nd October 2008, 23:46

Now that the instruction processing seems to work without problems and only very few traps still have to be explored it is time to start looking at events. The event passing and handling will be quite a challenge not only because it's barely documented but also because events are somewhat nondeterministic. It should also be the last thing left to do. :)

These are some useful informations which are given by the pdf:
The interface defines 9 callbacks (Events), which devices may invoke to notify content code of outside events, or to request specific actions
examples for possible events are successful media reads/writes, shutdown or media eject events, player security operations and idle events sent with each frame displayed
Event data is provided to content code in the Event Parameter area, which is a section of memory designated by content code.
response values are returned to the device via the same area

Observations of event handling we have already made:
at the end of the initialization phase the content code seems to signal the player it's readiness to process events with Trap_Finished()
after the execution of Trap_Finished() the watchdog counter is set (and after a break reset) to 0xFA0 so execution is interrupted in very short intervals (4000 DLX clock cycles)
the regular interruption of content code execution is used to check for events
if an event occurred and the execution is interrupted (watchdog reached <= 0) the following things happen:
parts of the memory space 0x000000-0x000FFF (that possibly is or very likely contains the "Event Parameter area") are modified
the program counter is set to 0x1000 (always?) after a copy of it's current value is stored in r28
the watchdog counter is reset to 0x7FFFFFFF which stops the regular interruption now that the content code is already processing an event
if no event occurred then only the watchdog counter is reset (to 0xFA0) and execution resumes normally
on the next call of Trap_Finished the following things happen:
the program counter is restored from r28
the watchdog counter is set (and after a break reset) to 0xFA0
this call likely signals the completion of the event processing to the player so the player will probably read the designated memory area 0x000000-0x000FFF to fetch the requested results
the memory space 0x000000-0x00003F seem to be special and is likely to hold meta data like parameters or the event id.

A great deal of the above observations were made by our anonymous contributers. :thanks:

Oopho2ei

4th October 2008, 01:37

I would like to introduce a new snapshot file format for our event analysis. The structure is shown below:

struct
{
uint32 pc;
uint32 if;
uint32 regs[32];
uint32 event_area[1024];
} snapshot_t

struct
{
snapshot_t snapshot0
snapshot_t snapshot1
} event_snapshot_file_t
Each file contains two snapshots: one snapshot before and one after the interruption of instruction processing (due to watchdog <= 0). All files which contain two identical snapshots are discarded. The remaining files are left for analysis.
As you can see each snapshot contains the program counter, the instruction filter, all 32 registers and most importantly the first 1000h bytes of vm memory (address space 0x000000-0x000FFF). The filenames have the format "epa_diff_%06d.bin".
You can write a program which analyses all the differences made by the player in the dedicated memory space (Event Parameter area) and finally creates a list/summary of all changes. Aim is to identify the section where the event id and the event parameters are stored and to create a list of all event types.

A snapshot package of that kind will be available soon.

Edit: This is a sample package: http://uploaded.to/?id=rfr3ew

Edit: That's the event snapshot package i have promised you: http://uploaded.to/?id=6ca19v

It seems every time i fast forwards or rewind a event is created. During the movie playback i have noticed about every 30s a new event.

Oopho2ei

5th October 2008, 02:12

There are still some other minor issues apart from event passing/handling. For example there is still no exception handling for cases like "division by zero", "arithmetic overflow" etc. The literature says a trap call would handle the exception but in the case of BD+ they seem to have defined the result of a division (both signed and unsigned) by zero to be zero. I had expected that our debugger would crash in this case but it didn't. Instead a message box with the text "/ by zero" is displayed. :)

I will look at the other cases soon and upload a list to the repository.

Edit: the documentation can be found here (http://svn.assembla.com/svn/bdplus/trunk/doc/instructions/exceptions.txt)

Oopho2ei

6th October 2008, 01:58

I have looked at the remaining Traps #0020 and #0530 (TRAP_DiscoveryRAM).

Trap#0020 has two parameters which are both vm memory addresses from which the player is reading a lot of data. I don't know what is done with the data yet. This is probably related to TRAP_Finished because both traps are in the same group. We still need a name for this trap which fits in alphabetically between TRAP_Finished and TRAP_MediaReadFile.

For TRAP_DiscoveryRAM the player fills a section of memory with some pattern and then loads the data which the content code is expecting from different places and thereby overwrites parts of the pattern again. I don't know if the pattern is really needed or just some sort of vendor specific countermeasure against memory analysis by malicious content code. The memory organization can be found here (http://svn.assembla.com/svn/bdplus/trunk/doc/player/memory_layout.txt) (work in progress)

schluppo

6th October 2008, 14:45

Implementation of trap_0x320 (TRAP_MemSearch) finished, works fine (for the few examples).
The title's VolumeID is now taken from a file called "volume_id.bin" which has to be in the same dir as the .jar. When trap_0x520(..,..,1,3) (TRAP_DeviceDiscovery) is called, the VolumeID is inserted into the result. Works fine for the two movies.

Things to code / fix:

- trap_20 (TRAP_EventGet??) is still untreated.
- trap_120 (TRAP_PrivateKey) is still untreated.
- trap_140 (TRAP_Sha) intermediate results are still wrong. I will fix this soon.
- trap_510 (TRAP_DeviceAccess?) is still untreated - there are just two calls of this trap in the 5000 snapshots for both movies. They both don't change memory.
- trap_520 (TRAP_DeviceDiscovery) - for parameters 1,3 there needs to be a timestamp in the result. Currently the timestamp is static, I will fix this soon.
- trap_530 (TRAP_DiscoveryRAM) - is still untreated, I am planning to look at this later today.
- trap_560 (TRAP_RunNative?) - there are no example calls for this yet.

Will upload new sources to the repository later today.

Edit: They are in the repository now. Also contain a try at TRAP_PrivateKey. Up to the end of the SHA-part, it works fine. Feel free to bugfix :)

Edit2: Timestamp added to the result of TRAP_DeviceDiscovery(...,...,1,3); Should be fine now.

Edit3: TRAP_DiscoveryRAM is now reading <len> bytes from a file called "RAM.bin", starting at <offset>. It then writes this data to <pDst>. It would be convenient to have a player memory-dump of the first 4mb to read from. Since that's probably not possible, I am working on a different solution now.

Oopho2ei

6th October 2008, 15:21

trap_20 (TRAP_EventGet??) is still untreated
I was told that there is no TRAP_EventGet (see posting #145). The event passing is likely done via a dedicated memory area (probably 0x000000-0x000FFF). The implementation of this trap is a bit strange. Some details can be found in posting #197
- trap_120 (TRAP_PrivateKey) is still untreated.We are still trying to figure out the curve parameters but i was told it works fine with the fixed (k,r) pair from posting #188.
- trap_520 (TRAP_DeviceDiscovery) - for parameters 1,3 there needs to be a timestamp in the result. Currently the timestamp is static, I will fix this soon.
I was told that TRAP_DeviceDiscovery needs player specific keys too. Do you see some kind of signature in the output of this trap? Or is it just the timestamp which is different? Otherwise it's likely that the public key (used by the content code to verify the signature created by TRAP_PrivateKey) can be found somewhere in the output of the trap. I was able to see that two different keys were accessed for qID == 1 and qID == 2 but no key seems to be necessary for qID == 3. Sadly i didn't check if those keys were really used somewhere. :rolleyes:
- trap_560 (TRAP_RunNative?) - there are no example calls for this yet.
And i hope there won't be an example. :)
Will upload new sources to the repository later today.
Have a look here (http://trac.assembla.com/bdplus/changeset/42) to see what i changed to make the data breakpoints work. Please only upload the source code (*.java files) to the repository because we have only 200MB disc space and all revisions are stored. When you upload a snapshot (to uploaded.to or whatever) could you please include a description of what files have to be present and in which directory? The output of "find *" will probably be enough.

Oopho2ei

7th October 2008, 13:36

There are no updates visible in the repository for the latest changes you announced. Also for some reason you have deleted all sources in revision 48 (http://trac.assembla.com/bdplus/changeset/48) and added new sources in revision 49 (http://trac.assembla.com/bdplus/changeset/49) and 50 (http://trac.assembla.com/bdplus/changeset/50). One revision later (51 (http://trac.assembla.com/bdplus/changeset/51)) you have finally added the description of what files are needed.

The result (revision 51) is correct but the way you work with the repository (it's not a ftp server!) has to be improved. First of all to upload your latest changes you need a single simple command:
svn commit -m 'this is a little description of what i have changed'
This command compares your source with the current (locally stored) snapshot of the repository and analyses the lines of code you have changed. It creates a patch (like this one (http://trac.assembla.com/bdplus/changeset/42?format=diff&new=42)) which it tries to apply to the real repository on the assembla server (which might have changed because other people applied their changes in the meantime). When you delete one of the source files from the repository and upload your own version you destroy the work of other people which have made updates to the files in the meantime. You should commit your changes every time you made an important change which leads to a still working implementation.

Feel free to bugfix :)
I think people are discouraged from doing that because the latest version of the repository is already outdated (you have made other changes):
Edit2: Timestamp added to the result of TRAP_DeviceDiscovery(...,...,1,3); Should be fine now.

Edit3: TRAP_DiscoveryRAM is now reading <len> bytes from a file called "RAM.bin", starting at <offset>. It then writes this data to <pDst>. It would be convenient to have a player memory-dump of the first 4mb to read from. Since that's probably not possible, I am working on a different solution now.
In posting #197 i've tried to explain that TRAP_DiscoveryRAM doesn't read parts of memory (security problems (access to secret keys etc) -> data would have to be relocated -> other player threads which access the previous address in the meantime might crash ....) instead depending on the address the data the content code expects is fetched from different places to build the requested section of player memory. This could be called "reading the virtual player memory".

Also you don't have to edit the postings to announce your changes. People can read the latest changes here (http://trac.assembla.com/bdplus/timeline). :)

Anyway thanks for your work and the updates. :thanks:

Edit: i make daily backups of TRAC and SNV so we can quickly move if necessary. But reverse engineering for interoperability (bd+ for linux) is normally considered legal so i doubt there will be any problems. :)

Oopho2ei

7th October 2008, 21:11

Ok, the last repository update looks fine. Now i would suggest we introduce some structure to the code. I have this (http://uploaded.to/?id=4nn0a3) in mind. You can already find the "bdsvm_player_interface.java" (unfinished) in the repository together with another new class "bdsvm_globals.java" which is supposed to be used for constants or global variables (if needed).

When i wrote the bdsvm_player_interface.java i had the BDVM.java still open and the autosave function seemed to have overwritten your latest patch. I have corrected this in revision 54. Sorry! :rolleyes:

Edit: The svg image can be displayed by your browser.

Oopho2ei

7th October 2008, 23:37

We were able to find all the curve parameters for the ECDSA algorithm used by TRAP_PrivateKey. They can be found in posting #188 (http://forum.doom9.org/showthread.php?p=1190187#post1190187).

You can now use any ECDSA implementation which fits your programming language. :)

Edit: The public key given in posting #188 can be found in the output of TRAP_DeviceDiscovery.

RunningSkittle

7th October 2008, 23:55

For the non technical readers following this, could you give an update to how things are progressing? :]

schluppo

8th October 2008, 04:29

Here (http://uploaded.to/?id=pvjwnc)'s a nice RAM.bin, if you put it in the same dir as the newest version of the debugger, this will take care of TRAP_DiscoveryRAM :)

This trap gives same result for same parameters and also same result for same parameters on both movies, so it seems like it really checks 'constant' areas of the virtual RAM. Most of the 'footprints' are by the way very similar to each other, maybe they are just offset to each other. I will investigate this when I have more time (could allow for a nicer treatment of this trap).

TRAP_SHA is fixed too (intermediate results were a pain in the *** for me).

Now only TRAP_PrivateKey remains to be treated for the first 5000 trap-calls and both movies. Will do that next...

Unguided execution stays in sync till trap call ~#500 (gets out of sync one trap call after TRAP_PrivateKey).

Edit: Sorry about the SVN-trouble, I was using a bad client and didn't think anyone would be bothered by it. The situation is fixed now.

Oopho2ei

8th October 2008, 13:56

For the non technical readers following this, could you give an update to how things are progressing? :]
The whole project consists of three major tasks:
1. vm instruction processing (95% done)
2. trap implementations (80% done)
3. event/callback processing (10 % done)

With the memory snapshots and traces (for the movies "The Day After Tomorrow" and "I Robot") from a licensed player we are able to debug and test parts of our implementations on real world examples.

All the keys we currently use for debugging will probably be revoked soon (if they have not been revoked already) so you probably won't be able to watch the latest BD+ protected movies with mplayer without new keys in the future.
The result of this project will hopefully be a working open source BD+ implementation which can be used together with linux and other free operating systems. I am hoping that DRM companies will look for bugs in our code to exploit so we can constantly improve the source until it finally meets the requirements of the secret BD+ specification.

I will investigate this when I have more time (could allow for a nicer treatment of this trap).
Have you (and other) looked at my idea of a modular BD+ implementation in posting #201? I believe it will be easier to understand and maintain the code. If you keep on squeezing everything into BDVM.java it will probably become even harder to "reorganize" the code later. Maybe bmnot can assist you and i am willing to help too. :)

FoxDisc

8th October 2008, 16:50

All the keys we currently use for debugging will probably be revoked soon (if they have not been revoked already) so you probably won't be able to watch the latest BD+ protected movies with mplayer without new keys in the future.

Thanks for the summary. Are the keys used by BD+ the same keys defined in the AACS spec, or do they use their own keys?

Oopho2ei

8th October 2008, 17:49

Thanks for the summary. Are the keys used by BD+ the same keys defined in the AACS spec, or do they use their own keys?
As for the ECDSA algorithm used by TRAP_PrivateKey we know that it uses different curve parameters (q,n, etc are different) so there is no relationship. I don't know about the 7 aes keys used by TRAP_Aes because they are all transformed (obfuscated) and nobody has bothered to reverse the transformation yet so we still use the corresponding transformed aes implementation. It's like using the keys without really knowing them. See postings #158 (http://forum.doom9.org/showthread.php?p=1187262#post1187262) and #163 (http://forum.doom9.org/showthread.php?p=1187564#post1187564) for details. It's some kind of whitebox attack resistant aes implementation. The basic idea to introduce random bijections which are applied at the end of one round and reversed before the beginning of the next round so to introduces a lot of noise to hide the underlying calculations and the round keys (including the key itself). Afaik unless those implementations don't use extremely large lookup tables they are all insecure. So that's the status on the keys. :)

Oopho2ei

9th October 2008, 02:06

I think qID =1 and qID=2 for TRAP_DeviceDiscovery might be the question for the primary and the secondary player certificate. Each certificate contains among other things the corresponding public key. You can find both public keys in posting #188 (http://forum.doom9.org/showthread.php?p=1190187#post1190187). The structure of each certificate could be similar to those in the AACS specification which you can find in this pdf (http://www.aacsla.com/specifications/specs091/AACS_Spec_Common_0.91.pdf) on page 42 (4.1 Drive Certificate).

I have furthermore analyzed the event snapshots i made (see posting #195 (http://forum.doom9.org/showthread.php?p=1191362#post1191362)). The differences are shown in this file (http://uploaded.to/?id=4zgboh). The following is an example of what was changed during the time the vm instruction processing was suspended (because watchdog <= 0):
PC = 000994C4 -> PC = 00001000
r28 = 000994D8 -> r28 = 000994C4
00000000 : 64DE004B E25BE65D 8D18BD98 14710E79 -> 00000220 00000000 00000001 00000057
00000020 : 96D3A639 B57F047B 6462D48B 7BFF5510 -> 00000000 00000000 6462D48B 7BFF5510
This shows you that the PC has been set to 0x1000 (a copy of the PC is stored in r28) and that the first 16 bytes of vm memory were replaced by "00000220 00000000 00000001 00000056". I currently think that 0x0220 is the event ID and then other 3 dwords are the parameters. When you study the file you will see, that the last parameter is incremented with every new event (which occurred about every 30s). If it suddenly jumps to a different value than this is because of me switching to a different scene in the movie :rolleyes:
Finally 8 bytes at address 00000020h are overwritten with zeros but i don't know why. :confused:

Assuming the value at 00000000h really is the event id then i have witnessed those events:
00000010 = occurred when i pressed the drive eject buttion, 1 parameter
00000110 = occurred at start of movie playback, 2 parameters
00000210 = occurred at start of movie playback, 2 parameters
00000220 = regular event (about every 30s), 3 parameters

I think i will find some answers when i follow the content code starting at 0x1000. :)

schluppo

9th October 2008, 03:35

The snapshot-packages contained some breaks too:

DAT v1.02:

... // Initialization etc.
TRAP_Finished;

break 0x110(0,0xFFFF);
TRAP_0x20(0,0x1000);
TRAP_Finished;

break 0x210(0,1);
TRAP_0x510(2,1,0x1EFB6C); // TRAP_DeviceAccess?
TRAP_0x510(0,1,0x1EFB6C); // TRAP_DeviceAccess?
TRAP_Finished;

break 0x110(0,1);
... // Lots of trap-testing here
TRAP_0x20(0xFCD9E,0xEFAF8);
TRAP_Finished;

break 0x010(0);

I Robot v1.02:

... // Initialization etc.
TRAP_Finished;

break 0x110(0,0xFFFF);
TRAP_0x20(0,0x1000);
TRAP_Finished;

break 0x210(0,1);
TRAP_0x510(2,1,0x3DFB70); // TRAP_DeviceAccess?
TRAP_0x510(0,1,0x3DFB70); // TRAP_DeviceAccess?
TRAP_Finished;

break 0x110(0,1);
... // Lots of trap-testing here
TRAP_0x20(0xFCBDA,0x2DFAFC);
TRAP_Finished;

break 0x010(0);

As you can see, the 'program' seems to be the same for both movies.

The player can tell the content code what to do by changing the first 20 bytes before it 'reactivates' the VM:
break 0x110(0,0xFFFF): ?
break 0x210(0,1): check whether disc is in drive and ready (?)
break 0x110(0,1): test bd+ system / player and get ready for playback
break 0x220(0,1,n): decrypt block <n>
break 0x010(0): shut down

Most interesting would be to see which traps are called after break 0x220(0,1,n) was invoked. The content code needs to get data, decrypt it and output it somehow. We don't know yet which traps are used for input and output (slots? 0x510? 0x20?).

Oopho2ei

9th October 2008, 13:08

Most interesting would be to see which traps are called after break 0x220(0,1,n) was invoked.
You can start implementing an event handler which writes the above data to the to the beginning of vm memory and sets registers pc and r28 appropriately. Then you only need to run your emulator and see what happens.
We don't know yet which traps are used for input and output (slots? 0x510? 0x20?).
Certainly not via slots. Probably through area 0x000000-0x000FFF and the results from the content code can maybe be passed to the player via Trap#0020.

schluppo

9th October 2008, 22:48

I looked a bit into the trap-call behaviour after different kind of breaks (all for DATv1.02).

When manually invoked after VM-initialization (i.e. after the first 5000 trap-calls), break 0x110(0,0xFFFF), break 0x210(0,1), break 0x110(0,1) and break 0x010(0) behave as expected.

Here's what I saw for break 0x220(0,1,n):

Depending on the value of <n>, several different traps are being checked (40-500 trap calls, some with bogus parameters, some with real parameters - they are being checked in exactly the same order as in break 0x110(0,1) - I found no new traps being called and there are no calls of known traps with new parameters).

I went through break 0x220(0,1,n) with 10 different values for n, and everytime, the last 5 trap calls in my log were identical:

IC: 75197 Trap #169: trap_0x410(0,14)= TRAP-SlotAttach
IC: 75203 Trap #170: trap_0x420(1ef964,0)= TRAP-SlotRead
IC: 75689 Trap #171: trap_0x520(1,1,22b48,1efa58)= TRAP_DeviceDiscovery
IC: 75789 Trap #172: trap_0x520(1,2,22b48,1efa58)= TRAP_DeviceDiscovery
IC: 78102 Trap #173: trap_0x10(ffffffff)= TRAP_Finished

By the way, everything before these 5 trap-calls is 'trap self-checking'.

So, the content code tries to read slot 0, but the current implementation of TRAP-SlotRead does not give the wanted result, hence the content code calls TRAP_DeviceDiscovery twice and then finishes (seems to be some kind of standard behavior in case that a trap gives a bad result).

Oopho2ei

9th October 2008, 23:48

Thanks for that information. So it looks like you need even longer snapshot packages now :rolleyes:

In the meantime could you check where the trap id is accessed in the content code and against what values it is compared to. Or maybe there is a lookup table which leads to the event id specific code. You can also record program counter traces for different event ids and compare them to see up to which point they are identical. The reason for this is simply to get a list of all event IDs that exist/handled by the content code. There are supposed to be 5 other event IDs (so 9 in total).

schluppo

10th October 2008, 02:33

From now on, I will write 'events' instead of 'breaks', whenever apropriate.

The code directly (first thirty instructions) after the break checks for equality of memory[0] (which holds <event_id>) with values from here:

0x22070: 18210200 80270200 FFFFFFFF 10020000
0x22080: 24250200 20020000 C4230200 10010000
0x22090: 60220200 00000000 18210200 10000000
0x220A0: 54260200 00000000 00000000 1000BD6F
0x220B0: 0C00DDE3 1000DD67 F8FFFEE3 00005DE0

(first from 0x2207C, then 0x22084 etc.)

If for instance <event_id> = 0x220, the according adress 0x223C4 is reached with IC = 47.

This indicates, that this version of BD+ has just the four events we saw so far (plus maybe event 0x0?).

Edit: Here's a small list of the traps which are called for the events in DATv1.02:

event 0x0: trap calls #1529-#2676 (the first 1528 trap calls are loading files and setting up the machine etc.).

event 0x10: execute just TRAP_Finished.

event 0x110(1): trap calls #2682-#5053.
event 0x110(2): trap calls #2682-#5053 (plus 4 additional TRAP_Random calls in the middle of testing TRAP_Random).
event 0x110(0xFFFF): execute TRAP_0x20 and then TRAP_Finished.

event 0x210: execute just TRAP_Finished (?)

Each of the following continues as stated in posting #211 after the testing is done:
event 0x210(1,1): trap calls #2682 - #2850 (= test TRAP_AddWithCarry).
event 0x210(1,2): trap calls #3760 - #3843 (= test TRAP_Random).
event 0x210(1,3): identical to event 0x210(1,1).
event 0x210(1,4): trap calls #3894 - #4001 (= test TRAP_AES).
event 0x210(1,5): trap calls #3473 - #3550 (= test TRAP_MemMove). Then custom test TRAP_MemMove.
event 0x210(1,6): trap calls #3551 - #3612 (= test TRAP_MemSearch). Then custom test TRAP_MemSearch.
event 0x210(1,7): trap calls #3142 - #3342 (= test TRAP_XorBlock).
event 0x210(1,8): identical to event 0x210(1,6).
event 0x210(1,9): identical to event 0x210(1,1).
event 0x210(1,a): trap calls #2851 - #3141 (= test TRAP_MultiplyWithRipple).
event 0x210(1,b): identical to event 0x210(1,5).
event 0x210(1,c): identical to event 0x210(1,2).
event 0x210(1,d): trap calls #4003 - #4442 (= test TRAP_DeviceDiscovery).
event 0x210(1,e): trap calls #4443 - #5049 (= test TRAP_DiscoveryRAM). Also trap calls #2070 - #2674 are the same.
event 0x210(1,f): trap calls #3614 - #3759 (= test TRAP_SHA).
event 0x210(1,10): identical to event 0x210(1,2).
event 0x210(1,11): identical to event 0x210(1,d).
event 0x210(1,12): identical to event 0x210(1,4).
event 0x210(1,13): identical to event 0x210(1,d).
event 0x210(1,14): trap calls #3344 - #3473 (= test TRAP_MemSet). Then custom test TRAP_MemSet.
event 0x210(1,15): identical to event 0x210(1,d).
event 0x210(1,16): identical to event 0x210(1,14).
...
event 0x210(1,0x56): custom test TRAP_MemMove.
event 0x210(1,0x85): trap calls #3844 - #3893 (= test TRAP_PrivateKey).
event 0x210(1,0xD3): identical to event 0x210(1,1).
event 0x210(1,0xE1): identical to event 0x210(1,f).
event 0x210(1,0x107): identical to event 0x210(1,d).
event 0x210(1,0x145): custom test TRAP_MemMove.
This should now cover all the trap-testing for Event 0x210. Note that it seems as if the current debugger gets past all of these tests (I logged all of this without snapshot- or PC/WD-guidance).

Oopho2ei

10th October 2008, 12:38

So the table is actually:
event ID | handler
--------------------
00000210 | 00022524
00000220 | 000223C4
00000110 | 00022260
00000000 | 00022118
00000010 | 00022654

So we got a new unknown event ID: 0. :)

I am planning to upload the new code base for the debugger soon (when it can be compiled without error/warning). I will help you finding the mistakes i made. Remember what i told you about the parameter checking: if the return value is wrong i get the trap parameters and the expected return value so i can fix the documentation :)

Edit: a possible terribly broken snapshot of the new codebase is in the repository. At least it compiles without errors. A new file "flash.bin" (link removed) is needed which contains all slots and 256 byte extra data (so 128kb).

Edit: the debugger with the new code base runs fairly stable now :) . There are still some bugs which will be sorted out soon. You need a new flash.bin (http://uploaded.to/?id=q55it2) because the one above had the the wrong byte order. A visible change for the user is the new monospace font and the leading zeros.

http://img101.imageshack.us/img101/7963/debuggerguixh1.th.jpg (http://img101.imageshack.us/my.php?image=debuggerguixh1.jpg)http://img101.imageshack.us/images/thpix.gif (http://g.imageshack.us/thpix.php)

I hope schluppo is not too mad at me about what i did to his code :D

schluppo

12th October 2008, 02:12

Looks quite fine.

One thing I noticed, is that you don't print out the value of trap-counter, instruction-counter or the trap-name, whenever a trap is executed. Also you always output 6 parameters even though a trap may have less parameters. I found the logging from previous versions more helpful when trying to quickly analyze which traps had actually been called during a run of the debugger.

Parameter Checking for TRAP_SlotRead is not correct - trap call #2037 is problematic:
IC: 37177656 Trap #2037: trap_0x420(1efd08,ffffffff)= TRAP-SlotRead

Parameter Checking for TRAP_AddWithCarry is not correct - trap call #2686 is throwing Array-Out-of-Bounds exception:
IC: 37506486 Trap #2686: trap_0x210(1ef928,3fffe0,8)= TRAP_AddWithCarry

Apart from that, I have no objections, nice work :)

Currently, I am idle. So, It would be great if you could provide a new snapshot-package, containing one or two calls of event 0x220.

Oopho2ei

12th October 2008, 10:42

One thing I noticed, is that you don't print out the value of trap-counter, instruction-counter or the trap-name, whenever a trap is executed. Also you always output 6 parameters even though a trap may have less parameters. I found the logging from previous versions more helpful when trying to quickly analyze which traps had actually been called during a run of the debugger.
I don't mind if you change it to something more useful. But please use System.out.format("bla = %08X, foo = %04d \n", bla, foo); or similar for console outputs.

Parameter Checking for TRAP_SlotRead is not correct - trap call #2037 is problematic:
IC: 37177656 Trap #2037: trap_0x420(1efd08,ffffffff)= TRAP-SlotRead

Yes and i have tried to fix it but without success. The slotid = 0xFFFFFFFF in TRAP_SlotRead means: read the currently attached slot. I have seen the check for slotid == 0xFFFFFFFF in the parameter checking code but didn't trace it into the following calls. My current guess is that something is wrong with the currently attached slot or there is no (valid?) slot attached.

Parameter Checking for TRAP_AddWithCarry is not correct - trap call #2686 is throwing Array-Out-of-Bounds exception:
IC: 37506486 Trap #2686: trap_0x210(1ef928,3fffe0,8)= TRAP_AddWithCarry
Oups. I must have missed that one.

Currently, I am idle. So, It would be great if you could provide a new snapshot-package, containing one or two calls of event 0x220.
I would suggest we fix the code of the debugger so it can run without snapshots and look at what the content code does for the events we create. For example TRAP_PrivateKey is still broken (i believe the problem is the conversion of the ASN.1 SEQUENCE (~46 bytes) in which the signature is represented by bouncycastle to the 40 byte raw hex data we have to place at 'dst'.
In a longer snapshot package you will probably only see the changes in the 0x00-0x30 memory area so i am not sure if this can help us. The above problem with TRAP_SlotRead shows me that there are still some things to study before we can turn to event handling/passing. Creating new longer snapshot packages is a full day of work for me. :rolleyes:

schluppo

13th October 2008, 01:42

I don't mean to sound negative in any way, but I saw a few things :) I found that TRAP_AddWithCarry is broken now.

#002694 TRAP_AddWithCarry (001EF928, 003FFFE0, 00000008);
Byte 1EF92C: snapshot: FF execute: FE
Byte 1EF92E: snapshot: FF execute: FE
Byte 1EF930: snapshot: FF execute: FE
Byte 1EF932: snapshot: FF execute: FE
Byte 1EF934: snapshot: FF execute: FE
Byte 1EF936: snapshot: FF execute: FE
Byte 1EF938: snapshot: FF execute: FE
Byte 1EF93A: snapshot: FF execute: FE
Byte 1EF93C: snapshot: FF execute: FE
Byte 1EF93E: snapshot: FF execute: FE
Byte 1EF940: snapshot: FF execute: FE
Byte 1EF942: snapshot: FF execute: FE
Byte 1EF944: snapshot: FF execute: FE
Byte 1EF946: snapshot: FF execute: FE

and so on. Looks like the carry value is not respected. Also the return value is still not correct:
#002732 TRAP_AddWithCarry (001ED740, 001EFAB0, 00000001);
Register error at trap number 2732:
post_reg_snapshot:
00000000 00000001 00000000 00000000 0002282C 00000004 001ED740 00000000 001EB738
00000001 001EFA70 00000000 001EF9B0 001EF9F0 001EFA30 001EF748 001EF970 001EF9D0
001EF92C 00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 000994B0 001EB6C4 001EFAD8 0005BF90
post_reg_execute:
00000000 00000000 00000000 00000000 0002282C 00000004 001ED740 00000000 001EB738
00000001 001EFA70 00000000 001EF9B0 001EF9F0 001EFA30 001EF748 001EF970 001EF9D0
001EF92C 00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 000994B0 001EB6C4 001EFAD8 0005BF90
Byte 1ED741: snapshot: 67 execute: 68
Byte 1ED742: snapshot: 89 execute: 88
Memory Error at trap 2732
(you need to return the carry flag).

Furthermore, parameter checking (or implementation?) is not correct any more for TRAP_MultiplyWithRipple:

#002852 TRAP_MultiplyWithRipple(001EF7B0, 00000004, 00000008, 00000000);
java.lang.ArrayIndexOutOfBoundsException: 35
at bdvm.bdsvm_player_traps.MultiplyWithRipple(bdsvm_player_traps.java:847)
at bdvm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.java:698)

Edit: Also, I thought that automatic loading of post-break-snapshots was convenient for testing the VM. Is there a reason to remove it already?

Oopho2ei

13th October 2008, 02:38

I don't mean to sound negative in any way, but I saw a few things :) I found that TRAP_AddWithCarry is broken now.

Furthermore, parameter checking (or implementation?) is not correct any more for TRAP_MultiplyWithRipple:

Those are the normal regressions when you make deeper changes in the code. I will fix it.

Edit: Also, I thought that automatic loading of post-break-snapshots was convenient for testing the VM. Is there a reason to remove it already?
Yes it is convenient and the code is not gone either (you can still load all different revisions we made from the repository). I didn't like the way it was implemented.

The signature generation in TRAP_PrivateKey seems to be working now with the Bouncycastle library. I have spent like 10 hours trying to figure out why the signature verification in the content code always failed. The reason was simply that in the Bouncycastle implementation "ECDSA" means = SHA-1 + ECDSA. What a waste of time :(

To install the library i have downloaded the bcprov-jdk16-141.jar (http://www.bouncycastle.org/download/bcprov-jdk16-141.jar) and placed the file in the directory "/usr/lib/j2sdk1.6-sun/jre/lib/ext". Furthermore i have added one line to the "java.security" located in the directory "/usr/lib/j2sdk1.6-sun/jre/lib/security" so it looks like this:
#
# List of providers and their preference orders (see above):
#
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=com.sun.net.ssl.internal.ssl.Provider
security.provider.4=com.sun.crypto.provider.SunJCE
security.provider.5=sun.security.jgss.SunProvider
security.provider.6=com.sun.security.sasl.Provider
security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.8=sun.security.smartcardio.SunPCSC
security.provider.9=org.bouncycastle.jce.provider.BouncyCastleProvider
The last line is new ^

You can see if the signature verification succeeded by looking at the console output of the debugger when executing the BDSVM files from "The Day After Tomorrow":
#000496 TRAP_Random (00002348, 00000010);
#000497 TRAP_Sha (001EF924, 001EFB24, 00000020, 00000003);
#000498 TRAP_PrivateKey (00000000, 00005514, 00002348, 00000010, 00000000);
#000499 TRAP_DeviceDiscovery (00000001, 00000001, 001EF794, 001EF78C);
#000500 TRAP_Aes (0005B430, 0005B430, 00000001, 0004B470, 00000000);
#000501 TRAP_DeviceDiscovery (00000001, 00000002, 001EFCC8, 001EFCC4);
If the verification failed then the above TRAP_Aes call is replaced by another TRAP_DeviceDiscovery call.

Currently the debugger seems to run correctly up to the first TRAP_Finished without the help of snapshots. At least i get the impression by just looking at the console output.

To those java experts out there: Is there a way to add the Bouncycastle libraries without modifying the java installation on the user side? We only need the ECDSA implementation.

schluppo

13th October 2008, 03:29

Works fine for me. I'm glad that TRAP_PrivateKey is working now :)

I let the debugger run without guidance till the first call of TRAP_Finished and a quick compare of the snapshots 'post_trap_reg_002643.bin' and 'post_trap_mem_002643.bin' with the contents of memory and registers after unguided execution revealed lots of differences. Anyway, since the trap-execution is in correct order, I just hope/assume that these differences are stemming from normal (legitimate) changes in the random input data.

Feeding a memory-snapshot which is signalling event 0x220(0,1,n) to the debugger still does not get past the first TRAP_SlotRead. I assume that the data which is read from slot 0 should be of a specific pattern (which the debugger does not know yet).

#000252 TRAP-SlotAttach (00000000, 00000014);
#000253 TRAP-SlotRead (001EF964, 00000000);
#000254 TRAP_DeviceDiscovery (00000001, 00000001, 00022B48, 001EFA58);
#000255 TRAP_DeviceDiscovery (00000001, 00000002, 00022B48, 001EFA58);
#000256 TRAP_Finished ();

Edit: Warning - I just let "I Robot v1.02" run without guidance and even though I do not have the big stream file (BDMV\STREAM\00001.m2ts) which is being hashed by the content code, and the hash-result hence must have been wrong, the debugger reached TRAP_Finish with just a very minor change in the sequence of executed trap calls: #2052 to #2084 were omitted (these trap calls were just checking the slot-trap's parameter checks). So it seems to be the case, that bad trap-results do not necessarily lead to immediate shut down of the VM. This makes it harder to see whether traps (especially the ones with nondeterministic result, since snapshots help for the other traps) really behave well.

Also invoking event 0x110(0,1) on the memory/registers after unguided execution of "DAT v1.02" gives a different order of trap calls than in the snapshot package, indicating, that there might have been a trap-call with bad result.

Oopho2ei

13th October 2008, 15:20

I have rewritten the code for TRAP_AddWithCarry and TRAP_MultiplyWithRipple. The site where we host our repository ( assembla.com ) currently seems to have technical difficulties so here is the patch (http://uploaded.to/?id=ka3h2p). I will commit the changes as soon as the problems are solved.

Edit: there was still a bug in TRAP_AddWithCarry which is fixed now. So above is a new patch (only one line has changed).

Edit: The problems at assembla.com seem to be solved now. The latest changes are in the repository :)

Edit: TRAP_SlotAttach is likely completely wrong. The patent states:
Using the address and the specified length, the procedure then computes a cryptographic hash (e.g., using SHA-1) of the code. If the hash result does not match the value of the authorization hash stored in the slot then slot zero is attached and an error is returned.
What you do is comparing the specified content code section with a database of all code sections you have seen so far (array "slotCode[][]"). Can you explain to me why?

Edit: TRAP_SlotRead( *, 0xFFFFFFFF ) is wrong. This call writes the three dwords "0x00000000, 0x00000000, 0x00000007" to dst and returns unless the currently attached slot number is zero in which case it returns STATUS_INVALID_PARAMETER. It looks like the entire slot handling has to be rewritten :(

schluppo

13th October 2008, 19:59

This happens about 20 times:
#002843 TRAP_AddWithCarry (001EBC60, 001EDC60, 0000000F);
Register error at trap number 2843:
post_reg_snapshot:
00000000 00000000 001EBC60 001EFC60 001EFC60 0002282C 0000000F 00000190
00000190 001EFAD0 0000001F 000079E0 001EBAD0 001EDAD0 00000020 00000080
00000200 000088B8 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EBA64 001EFAD8 00059488
post_reg_execute:
00000000 00000001 001EBC60 001EFC60 001EFC60 0002282C 0000000F 00000190
00000190 001EFAD0 0000001F 000079E0 001EBAD0 001EDAD0 00000020 00000080
00000200 000088B8 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EBA64 001EFAD8 00059488
Memory Error at trap 2843
Seems like the current version returns 1 in R[1] all the time, whereas it should return 0 in case that the carry flag is not set and 1 only in the case that the carry flag is set.

Also:
#002963 TRAP_MultiplyWithRipple(001EDBE4, 001EEB5C, 0000002F, E9DE141D);
java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at bdvm.bdsvm_player_traps.MultiplyWithRipple(bdsvm_player_traps.java:688)
at bdvm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.java:669)

Oopho2ei

13th October 2008, 20:53

Could you please post the values at src and dst so i can verify it? TRAP_AddWithCarry returns only '1' if the resulting number is one byte longer than both input numbers which can only happen if an overflow occurred.

You obviously have a modified version of the debugger (a branch). Why don't you upload your changes to the repository?

Edit: i found the problem. It's again the inability of java to tread unsigned numbers correctly. I have uploaded a workaround. Please report back if there are still issues with those traps. All our values are unsigned and java only knows signed numbers :rolleyes:

schluppo

13th October 2008, 23:53

I only re-added the automatic loading of post-break-snapshots so that I could let it run past the first TRAP_Finish again (otherwise there's no way to get to all those trap-tests). Since you did not like the way it was implemented, I did not want to add it to the repository again. Anyway, apart from that small change, I always use the newest version from the repository, especially I do not interfere with the implementation of traps or parameter checks.

TRAP_AddWithCarry and TRAP_MultiplyWithRipple are fine now. Next problematic one is this:

#003895 TRAP_Aes (001EE288, 001EE298, 00000000, 001EE238, FFF10000);
Register error at trap number 3895:
post_reg_snapshot:
00000000 00000000 00000000 00000000 0002282C 001EE298 001EE288 001EE238
001EE1D8 001EE248 001EE228 0003EC44 001EE198 001EE218 001EE158 00000200
00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EE0DC 001EFAD8 00062764
post_reg_execute:
00000000 80000001 00000000 00000000 0002282C 001EE298 001EE288 001EE238
001EE1D8 001EE248 001EE228 0003EC44 001EE198 001EE218 001EE158 00000200
00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EE0DC 001EFAD8 00062764

(bad parameter checking or bad treatmant of case OpOrKeyID=0xFFF10000)

another example:

#003899 TRAP_Aes (001EE288, 001EE248, 00000004, 001EE238, FFF10000);
Register error at trap number 3899:
post_reg_snapshot:
00000000 00000000 00000000 00000000 0002282C 001EE298 001EE288 001EE238
001EE1D8 001EE248 001EE228 00000000 001EE198 001EE218 001EE158 00000200
00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EE0DC 001EFAD8 00062978
post_reg_execute:
00000000 80000001 00000000 00000000 0002282C 001EE298 001EE288 001EE238
001EE1D8 001EE248 001EE228 00000000 001EE198 001EE218 001EE158 00000200
00000200 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EE0DC 001EFAD8 00062978
Byte 1EE288: snapshot: 3A execute: 00
Byte 1EE289: snapshot: D7 execute: 00
Byte 1EE28A: snapshot: 7B execute: 00
Byte 1EE28B: snapshot: B4 execute: 00
...
Byte 1EE2C4: snapshot: 04 execute: 00
Byte 1EE2C5: snapshot: 72 execute: 00
Byte 1EE2C6: snapshot: 5D execute: 00
Byte 1EE2C7: snapshot: D4 execute: 00
Memory Error at trap 3899

And then TRAP_DeviceDiscovery (probably bad parameter check again?):

#004006 TRAP_DeviceDiscovery (00000002, 00000000, 001EE854, 001EE7D0);
Register error at trap number 4006:
post_reg_snapshot:
00000000 00000000 001EF850 0000000F 0002282C 001EE854 00000009 00000000
001EE854 00000000 00000080 00000027 07D6060D 00000000 00000000 FFFFFFC4
07D6060D 00000000 001EE7D0 0005CC60 001EF814 001EFAD0 00000000 00000000
001EE854 00000000 00000000 00000000 000994B0 001EE760 001EFAD8 0006CBB4
post_reg_execute:
00000000 80000001 001EF850 0000000F 0002282C 001EE854 00000009 00000000
001EE854 00000000 00000080 00000027 07D6060D 00000000 00000000 FFFFFFC4
07D6060D 00000000 001EE7D0 0005CC60 001EF814 001EFAD0 00000000 00000000
001EE854 00000000 00000000 00000000 000994B0 001EE760 001EFAD8 0006CBB4
Byte 1EE854: snapshot: 00 execute: 5C
Byte 1EE855: snapshot: 00 execute: AA
Byte 1EE856: snapshot: 00 execute: 14
Byte 1EE857: snapshot: 52 execute: 80
...
Byte 1EE8D0: snapshot: 00 execute: E6
Byte 1EE8D1: snapshot: 00 execute: 3C
Byte 1EE8D2: snapshot: 00 execute: D8
Byte 1EE8D3: snapshot: 00 execute: 27
Memory Error at trap 4006

Here the endianness of the result needs to be changed:

#004007 TRAP_DeviceDiscovery (00000002, 00000001, 001EE8D4, 001EE7D0);
Byte 1EE8D4: snapshot: 00 execute: 2A
Byte 1EE8D7: snapshot: 2A execute: 00
Byte 1EE8DC: snapshot: 00 execute: 02
Byte 1EE8DD: snapshot: 2C execute: 08
Byte 1EE8DE: snapshot: 08 execute: 2C
Byte 1EE8DF: snapshot: 02 execute: 00
Byte 1EE8E0: snapshot: 00 execute: 03
Byte 1EE8E3: snapshot: 03 execute: 00
Byte 1EE8E4: snapshot: 00 execute: 01
Byte 1EE8E7: snapshot: 01 execute: 00
Memory Error at trap 4007

...

#004436 TRAP_DeviceDiscovery (00000001, 00000001, 001EF814, 001EF810);
Register error at trap number 4436:
post_reg_snapshot:
00000000 80000001 80000001 001EF7E8 001EF7E8 0002282C 001EF810 001EF814
00000005 0006DD30 00000000 0003EC54 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EF7E8 001EFAD8 0006D7E0
post_reg_execute:
00000000 00000000 80000001 001EF7E8 001EF7E8 0002282C 001EF810 001EF814
00000005 0006DD30 00000000 0003EC54 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EF7E8 001EFAD8 0006D7E0
Byte 1EF812: snapshot: 00 execute: 01
Byte 1EF813: snapshot: 2A execute: 1C
Byte 1EF814: snapshot: 07 execute: 00
Byte 1EF815: snapshot: D8 execute: 00
...
Byte 1EF92C: snapshot: 75 execute: FA
Byte 1EF92D: snapshot: F2 execute: F3
Byte 1EF92E: snapshot: 13 execute: 72
Byte 1EF92F: snapshot: 8A execute: 6F

OK, apart from these two traps (TRAP_Aes and TRAP_DeviceDiscovery), there is only the problem with

IC: 37177656 Trap #2037: trap_0x420(1efd08,ffffffff)= TRAP-SlotRead

remaining.

Oopho2ei

14th October 2008, 02:18

Two mistakes were in the parameter check documentation/implementation of TRAP_Aes and TRAP_DeviceDiscovery. I have already committed the patch to fix those issues. The big endian issue is fixed too (not committed yet).

#004436 TRAP_DeviceDiscovery (00000001, 00000001, 001EF814, 001EF810);
I don 't know why this call should fail. My best guess is that buf and len are too close together but i couldn't find and check for that but i should be able to find the problem soon. Some checks are done deep inside the trap implementation which is a bit more difficult to study.

OK, apart from these two traps (TRAP_Aes and TRAP_DeviceDiscovery), there is only the problem with

IC: 37177656 Trap #2037: trap_0x420(1efd08,ffffffff)= TRAP-SlotRead

remaining.
I think the current implementations for TRAP_Slot* are all wrong. Take a look at page 5 of the patent (http://uploaded.to/?id=za7x4t) to see how complex the slot handling actually is. I have already expressed my concern in posting #220 (http://forum.doom9.org/showpost.php?p=1196007&postcount=220).

schluppo

14th October 2008, 02:43

Referring to posting #220: Of course the hashing for slots still needs to be done. I have no information about how the slot-traps work on the inside, so I could only do what the snapshots showed, the 'security code' treatment hence is rudimentary. Never meant to create any other impression.

Apart from that, I noticed one more bug in TRAP_Aes:

#003977 TRAP_Aes (001EFA90, 001EFA91, 00000002, 001EFAC0, FFF10000);
Register error at trap number 3977:
post_reg_snapshot:
00000000 80000001 80000001 0002282C 001EFA90 001EFAC0 0002282C 10010000
00000002 00063B24 00000000 0003EC44 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EFA6C 001EFAD8 00061B60
post_reg_execute:
00000000 00000000 80000001 0002282C 001EFA90 001EFAC0 0002282C 10010000
00000002 00063B24 00000000 0003EC44 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EFA6C 001EFAD8 00061B60
Byte 1EFA90: snapshot: 00 execute: D1
Byte 1EFA91: snapshot: 00 execute: 08
Byte 1EFA92: snapshot: 00 execute: 6A
Byte 1EFA93: snapshot: 00 execute: 97
...
Byte 1EFAAC: snapshot: 00 execute: 2B
Byte 1EFAAD: snapshot: 02 execute: 4A
Byte 1EFAAE: snapshot: 29 execute: 48
Byte 1EFAAF: snapshot: DC execute: 48
Memory Error at trap 3977

#003978 TRAP_Aes (001EFA90, 001EFA9F, 00000002, 001EFAC0, FFF10000);
Register error at trap number 3978:
post_reg_snapshot:
00000000 80000001 80000001 0002282C 001EFA90 001EFAC0 0002282C 10010000
00000002 00063B24 00000000 0003EC44 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EFA6C 001EFAD8 00061BB4
post_reg_execute:
00000000 00000000 80000001 0002282C 001EFA90 001EFAC0 0002282C 10010000
00000002 00063B24 00000000 0003EC44 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 000994B0 001EFA6C 001EFAD8 00061BB4
Byte 1EFA90: snapshot: 00 execute: 8E
Byte 1EFA91: snapshot: 00 execute: 59
Byte 1EFA92: snapshot: 00 execute: 9F
Byte 1EFA93: snapshot: 00 execute: B9
...
Byte 1EFAAC: snapshot: 00 execute: 0C
Byte 1EFAAD: snapshot: 02 execute: 1F
Byte 1EFAAE: snapshot: 29 execute: 51
Byte 1EFAAF: snapshot: DC execute: D3
Memory Error at trap 3978

Oopho2ei

14th October 2008, 22:13

The TRAP_Aes parameter checking should be fine now. I had forgotten to copy an important check from my notes :rolleyes:
I am still not able to fix the checks for TRAP_SlotRead and TRAP_DeviceDiscovery though i made some observations. It's an illegal internal state which causes the failure of both calls. For TRAP_DeviceDiscovery i suggest we add a rule which lets the call fail if the "number of bytes written" pointer <len> points to a dword inside the output (which is written to <buf>). This has to be done on the trap implementation side because the interface (which does all the parameter checks) doesn't know the length of the output.

Regarding the slots i couldn't find any call of TRAP_SlotWrite which actually reads the data from <newContents>. Either all those calls i've checked have failed internally or the trap implementation of my player is wrong. I could see that slot 0 is read from non-volatile memory and that all 256 bytes are subsequently copied to the specified vm memory address.

Finally i suggest a command line interface for the debugger. The idea is to provide some basic commands to:
- set some environment variables (like where is the BDSVM or the snapshot directory)
- set/clear breakpoints (execution or data)
- execute n instructions or trap calls
- load a snapshot
- compare memory or register content with a snapshot
- dump or change parts of memory or registers
- record a instruction trace
- ...

All the snapshot stuff should be removed from the code. The user can run a script (maybe Tcl (http://www.usenix.org/publications/library/proceedings/tcl97/full_papers/lam/lam.pdf)?) which does all the work. :script:

Oopho2ei

15th October 2008, 11:56

Oopho2ei:

I've been reading your forum, reminds me of the good old 8-bit days and the reverse engineering I did back then. Anyway, I know nothing of java at all, but I certainly know C, and assembler.All my sources are OpenSource, and I must admit it is very tempting, but I am not sure where to start, or if the reference docs that you have done (?) that I would need are easy to get/read/understand. I know DVD and CSS fairly well (..now) but BD is new.
You can find the most important facts here (http://svn.assembla.com/svn/bdplus/trunk/doc/). A sample c(++) program is here (http://svn.assembla.com/svn/bdplus/trunk/bdvm.c).

The implementation consists of 3 basic steps:
1. implement the instruction processing (description of the instructions can be found here (http://svn.assembla.com/svn/bdplus/trunk/doc/instructions/list.txt) and search the web for the Hennessy and Patterson's DLX architecture)
2. trap processing (description of the traps can be found here (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/), in the patent (http://uploaded.to/?id=za7x4t) and in this thread)
3. event handling (still under research)
4. debugging (we use snapshots and traces from a licensed player which can be found here (http://forum.doom9.org/showpost.php?p=1183289&postcount=130))

If you have any questions first look at this document (http://securityevaluators.com/pdf/spdc_aacs_2005.pdf) and the patent (http://uploaded.to/?id=za7x4t). If you can't find an answer there post your question here.

schluppo

15th October 2008, 23:33

There seems to be a new variant of BD+ out: http://forum.slysoft.com/showthread.php?t=21177

This is a nice oppurtunity for me to list the previous 'versions' (currently there are at least 65 titles which employ BD+):

The first version was on just the first twenty or so BD+ titles (such as "Day After Tomorrow", "Fantastic Four: Rise of the Silver Surfer" etc., maybe on all the titles released in 2007?). This version did not use TRAP_MediaSHAFileHash so that it was possible to first copy a disc to an image with AACS removed but BD+ still intact and then mount or burn this image and play it with a licensed player.
The second version of BD+ fixed this bug and used TRAP_MediaSHAFileHash to verify that the inserted disc still contained AACS (actually TRAP_MediaSHAFileHash just verifies that small parts of a few files on the disc have a hash which is identical to a hardcoded byte-sequence in the content code). Apart from that, there don't seem to be much differences between the first and the second version of BD+. This version appeared early in 2008 and was used till around the middle of 2008. (Maybe with a minor update around March 2008, I think I remember AnyDVDHD treatment of the BD+-version on "Hitman" to be problematic for a while.)
The next version of BD+ appeared in June 2008 (first title was "Jumper" - AnyDVDHD was not able to handle this for a while). I don't have much information about what may have changed with this version of BD+.
The newest 'version' just appeared a few days ago on "Die Another Day". Rather than it being a really new version of BD+, it seems to me as if this is just the first title with BD+ where the main feature is split across several files and not stuffed into one big file. This could become important when implementing the way in which the data is passed to the VM in execution of event 0x220.

It would be interesting to get more information about the newer versions of BD+, such as having a trace of trap-calls for a movie as new as or newer than e.g. "Jumper".

On another note, I am currently idle but willing to do more trap-analyzing whenever any snapshots or similar are handed to me :) The debugger may currently not run with correct memory contents to the first TRAP_Finished on it's own (because of the well known TRAP-SlotRead(*,ffffffff) problem). Otherwise, it would be possible to really start treating the event handling, since we could try and just feed any event to the VM (after it has initialized itself on it's own in the first ~2600 trap-calls) and see what happens.

Oopho2ei

16th October 2008, 13:53

It's really amazing how you and gioowe can read the hidden messages in those slysoft forum postings. When i look through my crystal ball at the posting i only see that he had chocolate cakes this morning and that his grandma is turning 80 next week. Maybe i need to look at the least significant bits of the fox image in the top left corner :rolleyes:

Seriously we should focus on the correct implementation of all traps before we start anything new. When i started looking at events i was under the impression that the trap implementation is almost complete (e.g. with the details from the patent) and the code is in a good condition but i was really wrong. There is still a lot to be done.
I am currently looking at TRAP_SlotAttach and TRAP_SlotWrite will be next. In the meantime you should review the code and correct the short trap descriptions you can find in bdsvm_player_traps.java. I will upload new snapshots when i think it is appropriate.

Oopho2ei

16th October 2008, 15:25

This is a description of how TRAP_SlotAttach works. The example below is for the first call of TRAP_SlotAttach in "The Day After Tomorrow":

call: TRAP_SlotAttach( slot = ?, codeLen = 0x0000000C )
PC = 0001AC70h (already incremented)

That's the data which is hashed with SHA-1:
00 01 ac 6c 00 00 00 00 ; PC-4 ( & 0x3FFFFC ) and instruction filter (IF)
6f bd 00 1c e0 3d 00 08 e3 fd 00 04 d4 20 00 00 ; 16 bytes from 0x1000
e4 00 04 10 58 1f 00 00 e4 00 04 20 58 1f 00 00 ; #codeLen dwords from PC-4
e4 00 04 30 58 1f 00 00 e4 00 02 30 58 1f 00 00
e4 00 80 10 58 1f 00 00 e4 00 80 20 58 1f 00 00
The resulting SHA-1 hash:
h = 78f623b22178c6150c420a3bff949c85ae7f75bd
SHA-1 hash from hash:
h' = 3ac6e39ffddc1c66429aae0600c61bf0ec340d8a
This is the resulting hash to be compared against the hash stored in the slot. I have no idea why the hash is hashed again (makes no sense to me). The offset of the hash within the slot is probably 0x30. I couldn't reliably determine the beginning and the end of the slot so the address (0x30) is a good guess based on some assumptions.

Edit: the unknown dword is the instruction filter (IF)

tteich

16th October 2008, 15:57

This is a description of how TRAP_SlotAttach works. The example below is for the first call of TRAP_SlotAttach in "The Day After Tomorrow":

call: TRAP_SlotAttach( slot = ?, codeLen = 0x0000000C )
PC = 0001AC70h (already incremented)

That's the data which is hashed with SHA-1:
00 01 ac 6c 00 00 00 00 ; PC-4 ( & 0x3FFFFC ) and unknown dword
6f bd 00 1c e0 3d 00 08 e3 fd 00 04 d4 20 00 00 ; 16 bytes from 0x1000
e4 00 04 10 58 1f 00 00 e4 00 04 20 58 1f 00 00 ; #codeLen dwords from PC-4
e4 00 04 30 58 1f 00 00 e4 00 02 30 58 1f 00 00
e4 00 80 10 58 1f 00 00 e4 00 80 20 58 1f 00 00
The resulting SHA-1 hash:
h = 78f623b22178c6150c420a3bff949c85ae7f75bd
SHA-1 hash from hash:
h' = 3ac6e39ffddc1c66429aae0600c61bf0ec340d8a
This is the resulting hash to be compared against the hash stored in the slot. I have no idea why the hash is hashed again (makes no sense to me). The offset of the hash within the slot is probably 0x30. I couldn't reliably determine the beginning and the end of the slot so the address (0x30) is a good guess based on some assumptions.

Perhaps they apply a Hash chain as authentication mechanism? http://en.wikipedia.org/wiki/Hash_chain

Oopho2ei

16th October 2008, 19:07

Perhaps they apply a Hash chain as authentication mechanism? http://en.wikipedia.org/wiki/Hash_chain
I don't know how a valid slot looks like so it might be possible that h' wasn't compared against the hash from the slot but against the hash from the hash from the slot. So you could be right. It makes me wonder how the entire system looks like if there is an insecure channel between the slot memory and the BD+ core. Maybe the slot data is stored inside a smartcard? It would also mean that the content code can only successfully attach to this slot twice.

Some details about TRAP_SlotWrite:

TRAP_SlotWrite(UINT8 *newContents);

p = some address inside the player (volatile) memory

- copy 0x10 bytes from newContents+0x20 to p
- copy 0x14 bytes from newContents+0x30 to p+0x10
- copy 0xB4 bytes from newContents+0x4C to p+0x2C
So the data at
newContents+0x00 ... newContents+0x1F and
newContents+0x44 ... newContents+0x4B is ignored and could be reserved memory (eg. for the hash) inside the slot. Altogether it adds up to 256 which is supposed to be the size of a slot.

Oopho2ei

16th October 2008, 21:48

This is what i assume is the structure of a slot. The example data is from slot 0 (DAT).

Addr : Data - Description
-----------------------------------------------------------------------------------
0000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - Creator Media ID ?
0010 : 1d 2f c6 74 62 fd f9 86 35 7a 8f 80 8f a1 29 8b - Last Update Media ID ?
0020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - Slot Private Data ?
0030 : a9 78 8b 0b b2 8d e0 e1 3f 40 af de e9 a3 31 6b 25 3c fa 21 - Attach Authentication Hash
0044 : 00 00 00 00 - ?
0048 : 00 00 00 8e - Last Update Sequence Counter ?
004C : d4 8e 75 57 c6 e6 0b 46 a7 14 60 99 46 ff 65 0d \
005C : fe e5 48 8a ee dd 48 ad dd 47 8d 1d 87 53 d7 04 |
006C : 59 5b 6d b1 57 b6 04 d9 aa 32 da c6 ff 9b fe 02 |
007C : 1f b2 1b 8e 5b 61 88 0d 85 8e 72 d1 5c c7 6f f2 |
008C : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |
009C : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff } Data Payload (180 bytes)
00AC : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |
00BC : ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |
00CC : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |
00DC : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |
00EC : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |
00FC : 00 00 00 00 /
Schluppo could you please start implementing the slot traps from the above notes so i can focus on research? :)

schluppo

17th October 2008, 00:12

I changed TRAP-SlotAttach according to posting #230. For the first call in "DAT v1.02", this gives the same h' as in the posting, so it seems to be fine. Then the new code compares the 20 bytes that are h' with the 20 bytes starting at 0x30 in slot number <slot> (where <slot> is the first parameter of the trap). If they are equal, the global variable AttachedSlot is set to <slot> and STATUS_OK is returned, otherwise AttachedSlot is set to 0 and STATUS_INVALID_PARAMETER is returned.

About TRAP-SlotWrite: What does the trap do if there is no slot attached? And since you changed the default-value of AttachedSlot (meaning 'no slot attached') to 0, what happens, if slot 0 is supposed to be attached?

tteich

17th October 2008, 13:33

I found an interesting patent which explains a bit how the different .svm files are used: http://www.freepatentsonline.com/EP1841120.html

00000.svm seems to contain code common to all players, and is supposed to select a different (player dependent) xxxxx.svm for known players, or a common file for all unknown players. The selected file contains security checks and content scrambling code.

Oopho2ei

17th October 2008, 15:25

I changed TRAP-SlotAttach according to posting #230. For the first call in "DAT v1.02", this gives the same h' as in the posting, so it seems to be fine. Then the new code compares the 20 bytes that are h' with the 20 bytes starting at 0x30 in slot number <slot> (where <slot> is the first parameter of the trap). If they are equal, the global variable AttachedSlot is set to <slot> and STATUS_OK is returned, otherwise AttachedSlot is set to 0 and STATUS_INVALID_PARAMETER is returned.
Thanks. I have made some changes on your code. Could you please run your tests again and see if everything is still alright? :rolleyes:

About TRAP-SlotWrite: What does the trap do if there is no slot attached? And since you changed the default-value of AttachedSlot (meaning 'no slot attached') to 0, what happens, if slot 0 is supposed to be attached?
I am trying to figure this out now.

I found an interesting patent which explains a bit how the different .svm files are used
Thanks a lot. I think it explains even more. Look at page 40 (FIG. 11) of the pdf version (http://uploaded.to/?id=1dn4ix). This is probably the player certificate which we get from TRAP_DeviceDiscovery with parameters dev = 1 (player?) and qID = 1 and 2. The first dword is the length which is obviously correct.

Edit: From this patent i can also tell that the 'S' in BDSVM stands for "Secure" (and not "Secret" like i thought).

Edit: FIG.5 (p.34) and FIG.6 (p.35) illustrate the event handling and the final correction of broken segments in the decrypted video and audio stream. The process is described starting with page 13. To see the differences in the output for "The Day After Tomorrow" look at posting #13 (http://forum.doom9.org/showpost.php?p=1174977&postcount=13).

schluppo

17th October 2008, 18:16

Your changes are fine, as far as I can see. :)

Oopho2ei

17th October 2008, 19:04

Your changes are fine, as far as I can see. :)
Good! ;)

Some scenarios which should give you an idea how the AttachedSlot mechanism works.

- the first trap which is executed is TRAP_SlotWrite:
-> writes to slot 0

- slot 0 is attached ( TRAP_SlotAttach( 0, 4 ) finished successfully ):
-> the following TRAP_SlotWrite writes to slot 0

- TRAP_SlotAttach( 0x1F3, 0c0C ) succeeded and the following trap "TRAP_SlotAttach( 0x1F3, 0c0C )" failed ( hash missmatch ):
-> the following TRAP_SlotWrite writes to slot 0

- TRAP_SlotAttach( 0x1F3, 0c0C ) succeeded:
-> TRAP_SlotWrite writes to slot 0x1F3
-> another TRAP_SlotWrite does not write to slot 0 (so probably writes to slot 0x1F3 as well)
-> the following TRAP_SlotRead calls read the contents of slots 0 (unchanged) and 0x1F3 correctly
If you need more information just ask.

Edit: As you might have noticed the (Creator/Last Update) Media ID in the Slot is the Volume ID. For "The Day After Tomorrow" it is 1D2FC67462FDF986357A8F808FA1298B.

Edit: i've added a new class to handle the slots. All changes can be viewed here (http://trac.assembla.com/bdplus/timeline) as usual.

Accident

17th October 2008, 23:35

The DLX reference documentation mention that R0 is always set to 0 and any writes are ignored. Is this the case here as well, should I force R0 read-only?

Oopho2ei

18th October 2008, 01:36

The DLX reference documentation mention that R0 is always set to 0 and any writes are ignored. Is this the case here as well, should I force R0 read-only?
Good question. All i know is that the licensed player i study has memory reserved for r00. I have placed a check in our java debugger to see if r00 has at any point a value different from zero but it seems this is never the case. So at least the compiler doesn't use it to store values. I suggest you output a warning when it's different from zero.

Some news about TRAP_SlotAttach and TRAP_SlotRead with parameter slot=0xFFFFFFFF. When TRAP_SlotRead is called with slot=0xFFFFFFFF it writes 3 dwords (DW0 DW1 DW2) to dst. DW1 and DW2 are two bytes B1 and B2 which were expanded to UINT32 and DW0 seems to be originally a 16 or 32 bit value. TRAP_SlotRead only seems to read and expand those values but TRAP_SlotAttach really changes them if called with slot = 0xFFFFFFFF which is supposed to be the way a new slot is allocated.

The output of TRAP_SlotRead before TRAP_SlotAttach (both with slot = 0xFFFFFFFF):
00000000 00000000 00000007

Looking at the source for DW0, DW1 and DW2 the following things happen now when TRAP_SlotAttach is called with slot = 0xFFFFFFFF:
if ( B2 > 2 )
{
B2 = B2 - 1;
}
B1 = B2;
Initially B1=0 and B2=7 so the result is B1=6 and B2=6. Also DW1 now contains a slot number. When TRAP_SlotRead is called now (with slot = 0xFFFFFFFF) the result is:
000001F2 00000006 00000006
When i try to read the slot 0x1F2 now with TRAP_SlotRead i get the output:
0000:0010 1d 2f c6 74 62 fd f9 86 35 7a 8f 80 8f a1 29 8b
0000:0010 1d 2f c6 74 62 fd f9 86 35 7a 8f 80 8f a1 29 8b
0000:0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0040 00 00 00 00 00 00 00 00 00 00 01 15 00 00 00 00
0000:0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000:00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
So the TRAP_SlotAttach did create a new slot and attached it as well (otherwise we would get 256 zeros). The "Creator Media ID" (first 16 bytes) and the "Last Update Media ID" are both set to the AACS Volume ID. For some reason the "Last Update Sequence Counter" already contains a value 0x00000115.

Now when i again call TRAP_SlotAttach followed by a call of TRAP_SlotRead (both with slot=0xFFFFFFFF) i get the result:
000001F1 0000005 00000005

So from these observations i can tell that TRAP_SlotAttach really allocates a new slot when called with parameter slot = 0xFFFFFFFF. But TRAP_SlotRead does not read the currently attached slot when called with parameter slot = 0xFFFFFFFF as described in the patent. Instead it returns three dwords which seem to describe what slot is currently attached and how many slots have been allocated or still can be allocated . Or something similar :rolleyes:

Edit: If attached successfully with TRAP_SlotAttach( 0x1F3, 4 ) then the output if TRAP_SlotRead( 0x100, 0xFFFFFFFF ) is: 000001F3 0000000 00000007
So this leaves now little doubt that the first dword always is the currently attached slot identifier.

bshep

18th October 2008, 02:05

So from these observations i can tell that TRAP_SlotAttach really allocates a new slot when called with parameter slot = 0xFFFFFFFF. But TRAP_SlotRead does not read the currently attached slot when called with parameter slot = 0xFFFFFFFF as described in the patent. Instead it returns three dwords which seem to describe what slot is currently attached and how many slots have been allocated or still can be allocated . Or something similar

This is just a wild guess but seems to fit:
DW0 = last allocated or currently attached slot ( you can test this by attaching a different slot and seeing if reading from slot 0xFFFFFFFF still give the last allocated slot or the currently attached slot )
DW1 and DW2 = seems to mean the number of slots available, would be interesting to see what happens if they are 0 and you try to allocate a new slot, if this asumption is true then 8 slots are avaiable to be allocated.

Edit: Just saw your edit, what does the test imply about DW1 and DW2, how many slots were already allocated in that test?

Accident

18th October 2008, 06:04

Sorry for the noise, but I have now got it running to this point:

[dlx] 0001A6F8 (I=18):ADDIE(R1 = R0 + 0007): 7 = 0 + 7
[dlx] 0001A6FC (I=38): SDW ([R29 + 10] = R1 ): 001EFF50 = 00000007
[dlx] 0001A700 (I=2D): JAL (R31 = 0001A704 ; PC += 04E8): 0001ABEC = PC + 1256
[dlx] 0001ABEC (I=39): TRAP([110]): TRAP 272
[dlx] 0001ABF0: TRAP 00000110 (272)
[dlx] TRAP_Aes()
[bdtest] executed 417 instructions

I am not sure it is all correct, but to verify I need to add TRAP_Aes first, since the snapshot is taken after the trap (correct yes?)

Since there is no stack, how are the arguments to TRAP_Aes passed? (or any trap for that matter).

(Edit1)
Ok, the param_check has details for at least Aes.
Here (http://svn.assembla.com/svn/bdplus/trunk/doc/traps/param_check/0110.txt)

(Edit2)
Although, not what to do with return code, but I will assume R1 as per DLX specification.

(Edit3)
My first trap to Aes return INVALID_PARAMETERS. So I need to do some debugging

Oopho2ei

18th October 2008, 12:35

Edit: Just saw your edit, what does the test imply about DW1 and DW2, how many slots were already allocated in that test?
Well DW2 is decremented each time a new slot is allocated. It's initial value is 7 so no new slots were allocated at that time. I am not sure when happens when i try to allocate a new slot and DW2 = 2. My guess about DW1 is that it shows the number of the private slot that is currently allocated if you think of a few (~ 6) consecutive slots as private slots. DW2 seems to limit the number of additional slots the content code can allocate in in one session (when watching the movie once). I have to run further tests. If you have read something in the patents which matches above observations please tell me.

Sorry for the noise, but I have now got it running to this point:
You should use the post-trap snapshots for all trap calls at the moment. You don't need to implement the parameter checking or the actual trap implementation yet. The stack pointer is (convention) r29 and the return value is stored in r01. Also you maybe don't want to send a message to the console for every instruction. When the initialization of the content code is complete (first call of TRAP_Finished) you will have executed millions of instructions and several thousands trap calls. :rolleyes:
You should first make sure that the instruction processing is working by comparing your program counter, instruction register and the timer with the traces given in the packages you can find in posting #130 (http://forum.doom9.org/showpost.php?p=1183289&postcount=130). If you fixed all the bugs (if any) you should proceed implementing the parameter checking for all traps and verify with the post trap register snapshot that r01 was set correctly.

Edit: You have probably missed to look at these (http://svn.assembla.com/svn/bdplus/trunk/doc/dlx/registers.txt) notes. ;)

Accident

18th October 2008, 12:55

You should first make sure that the instruction processing is working by comparing your program counter, instruction register and the timer with the traces given in the packages you can find in posting #130

Can I confirm that 000000.dat is AFTER the first trap executes.

I can use the register.dat to confirm my instructions, sans R1 return code. So I will do that right now.

It only prints all instructions if I ask it to, so far I just tell it to run " 1 trap" and then it stops. I have a CVS, but what is the deal with "secrecy"? Is there need to stay anonymous? (Referring to Jon and CSS)

I had read those notes and forgot, I was following the wikipedia notes on DLX more :) My bad, apologies. Also, your instructions list for arithmatic shift has "see note below" without a note :), but the wiki docs had it.

-----------------------------

Ok so the result is as expected, I guess it calls Aes with op=7 to test it. Please let me know if I am too noisy.

[dlx] 0001ABEC (I=39): TRAP([110]): TRAP 272
[dlx] ** TRAP 00000110 (272) return: 0001ABF0
[dlx] TRAP_Aes(): 80000001
[dlx] getPC (0001ABF0)
[bdtest] reached TRAP at 418: PC=0001ABF0.
[dlx] Comparing registers with 'dat/post_trap_snapshots/post_trap_reg_000000.bin'
Reg : VM Regs == Loaded Regs
R0 : 00000000 == 00000000 ; OK
R1 : 80000001 == 80000001 ; OK
R2 : 001EFF6C == 001EFF6C ; OK
R3 : 00005550 == 00005550 ; OK
R4 : 00005570 == 00005570 ; OK
R5 : 00005560 == 00005560 ; OK
R6 : 00017A50 == 00017A50 ; OK
R7 : 00017A4C == 00017A4C ; OK
R8 : 00000000 == 00000000 ; OK
R9 : 00000000 == 00000000 ; OK
R10 : 00000000 == 00000000 ; OK
R11 : 00000000 == 00000000 ; OK
R12 : 00000000 == 00000000 ; OK
R13 : 00000000 == 00000000 ; OK
R14 : 00000000 == 00000000 ; OK
R15 : 00000000 == 00000000 ; OK
R16 : 00000000 == 00000000 ; OK
R17 : 00000000 == 00000000 ; OK
R18 : 00000000 == 00000000 ; OK
R19 : 00000000 == 00000000 ; OK
R20 : 00000000 == 00000000 ; OK
R21 : 00000000 == 00000000 ; OK
R22 : 00000000 == 00000000 ; OK
R23 : 00000000 == 00000000 ; OK
R24 : 00000000 == 00000000 ; OK
R25 : 00000000 == 00000000 ; OK
R26 : 00000000 == 00000000 ; OK
R27 : 00000000 == 00000000 ; OK
R28 : 00001160 == 00001160 ; OK
R29 : 001EFF40 == 001EFF40 ; OK
R30 : 001EFF84 == 001EFF84 ; OK
R31 : 0001A704 == 0001A704 ; OK
[bdtest] executed 418 instructions, 1 trap

Having fun so far..

Oopho2ei

18th October 2008, 13:42

I have a CVS, but what is the deal with "secrecy"? Is there need to stay anonymous? (Referring to Jon and CSS)
Could you register at www.assembla.com and tell me your nickname so i can add you to the project. I make daily backups.
The goal of this project is to support BD+ in linux and other open source operating system so you can watch your movies without proprietary software on your pc. Reverse engineering to achieve interoperability is normally considered legal although patent owners usually try to fight this so you should stay anonymous.
Both AACS and BD+ were designed with renewability in mind so the system can quickly recover from a player compromise and leaked key material which is a major advantage over css. The key material which i provide for debugging will very likely soon be revoked so this implementation will very likely stop working for newly released movies. So this project is no threat to the security of the system. But both AACS and BD+ can only renew protection for movies released after the compromise.
Finally Slysoft has successfully reverse engineer BD+ a while ago and they sell their product commercially.

Please let me know if I am too noisy.
It's the purpose of this thread to guide the implementation efforts. It's likely that more people have the same questions. If you don't spam the thread with new postings it's fine. You can also write me private messages.

Edit: revision 86 now has experimental support for proper slot handling. @Schluppo and others: please run some tests :)

schluppo

18th October 2008, 18:13

Still:
#002037 TRAP-SlotRead (001EFD08, FFFFFFFF);
Register error at trap number 2037:
post_reg_snapshot:
00000000 00000000 00000000 10020000 00000101 00000001 0002282C 001EF9DC
001EFADC 00000100 00000000 0003E01C 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00001160 001D02CC 001EFD1C 0007C4D4
post_reg_execute:
00000000 80000001 00000000 10020000 00000101 00000001 0002282C 001EF9DC
001EFADC 00000100 00000000 0003E01C 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00001160 001D02CC 001EFD1C 0007C4D4
Byte 1EFD09: snapshot: 00 execute: 02
Byte 1EFD0A: snapshot: 00 execute: 28
Byte 1EFD0B: snapshot: 00 execute: 2C
Byte 1EFD13: snapshot: 07 execute: 00
Memory Error at trap 2037
All other calls of TRAP-Slot* behave as in the snapshots ("DAT v1.02") with the revision 86 of the debugger.

Oopho2ei

18th October 2008, 19:40

#002037 TRAP-SlotRead (001EFD08, FFFFFFFF);
All the time is was looking for a reason why this call should fail and why our debugger lets it execute successfully. :stupid:
The constant 0xFFFFFFFF gets signed extended when it is compared with a variable of type long. I made the same mistake several times so the last patch (to revision 87) fixes multiple bugs. Could you please check how the debugger behaves now after TRAP_Finished when you pass events (eg. write the event id and parameter manually to 0x00-0x030)?

schluppo

18th October 2008, 20:52

OK, until the first TRAP_Finished, only TRAP_Random, TRAP_DeviceDiscovery(1,3) and TRAP_PrivateKey give different results than the snapshots (due to 'random' data in these results).

I ran the debugger without guidance to the first break and then invoked some events with following results:

event 0x110(0,0xFF000) - TRAP_0x20 and then TRAP_Finished are executed. This event behaves as in the snapshot-package.
event 0x210(0,1) - Just TRAP_Finished is executed. This event behaves NOT as expected.
event 0x110(0,1) - Executes the whole bunch of trap testing. This event behaves as in the snapshot-package.
event 0x220(0,1,n) - Just TRAP_Finished is executed. This even behaves NOT as expected.

So, why do some events behave differently than in the snapshots?

1) Either there is an unnoticed fault/bug in the trap-treatment leading to a 'bad' state of memory when the first TRAP_Finished is reached. This bug could only be in the three traps which are introducing random data (since the other traps behave as in the snapshots).
2) Or the behaviour of the events could depend on the order in which they are executed. This would mean, that for instance event 0x220(0,1,n) can not be invoked unless event 0x110(0,1) has happened earlier. It seems to be somehow reasonable that BD+ first wants the traps to be tested before it is willing to decrypt movie data. Nevertheless, I tried and first let the debugger run to the first TRAP_Finished and then fed event 0x110(0,0xFF000) to the code. Then I invoked event 0x210(0,1) on the resulting state. This is the same order of events as in the snapshots and still event 0x210(0,1) behaved bad.
3) I may have made a mistake while doing these tests. I will double-check my method of event-invoking now.

Oopho2ei

18th October 2008, 21:04

Thanks. I will make a longer snapshot package now. In the meantime please look at figure 5 and 6 in this (http://uploaded.to/?id=1dn4ix) document and read the corresponding text (if you haven't done that already).

Edit: A new package (v1.03) for "The Day After Tomorrow" is available now: http://uploaded.to/?id=5dwn30

I compressed the snapshots with the diffarchive tool i introduced a few weeks ago. It was written by one of our anonymous contributers and manages to shrink the 50GB of snapshots to less than 20MB. There is no need to extract the archives on disc although the tool provides you with that option.

For every archive i've included the first snapshot so you can test if you handle the archive properly. The first snapshot is always compared against the empty snapshot (filled with zeros). A description about the archive structure can be found here (http://svn.assembla.com/svn/bdplus/trunk/diffarchive/). I also wrote a small readme which contains the syntax in EBNF.

@Schluppo: i am currently implementing support for the archives in the debugger as a new class "snapshots". Four instances (objects) of this class "snapshot_trap_mem", "snapshot_trap_reg", "snapshot_break_mem" and "snapshot_break_reg" for every snapshot type we have. The constructor gets the filename on instantiation and eg. with "snapshot_break_reg.next()" would you switch to the next snapshot. The current snapshot memory can be accessed with eg. "snapshot_break_reg.memory". I hope it get it working soon you can start your analysis of the event handling :)

Accident

19th October 2008, 05:21

It was going well, until trap milestone 484 showed I forgot instruction 0x1E. :)

Are there any saves of what the PC should be at each post_trap_mem milestone? My program runs on its on, and the registers all compare correctly (except R1 from the trap), it fixes R1, loads the next core and resumes.

But somewhere along the line the PC becomes very repetitive, until eventually it gets stuck on trap PC=0001AC74 (looping back to the trap). It stays here until post_trap_mem_002029.bin where PC goes really high (somewhere in the 3GB area, handled with the address masking, but there are only zero's there). So I must be doing something wrong earlier perhaps, which could be that my PC is lost.

I also do nothing with WD yet, apart from initialising it. I assume I should still be able to finish all the post_trap_mem milestones?

Boring output:

[bdtest] reached TRAP at 37019913:2026 PC=0001AC74.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002026.bin'
[bdtest] reached TRAP at 37019929:2027 PC=0001AC74.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002027.bin'
[bdtest] reached TRAP at 37021630:2028 PC=0001AC04.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002028.bin'
[trap] trap_DeviceDiscovery()
[bdtest] reached TRAP at 37023357:2029 PC=0001ABFC.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002029.bin'
[dlx] FE80103D (I=2C): J (PC += FFFFFFFC): FE80103D = FE801041 + FFFFFFFC
[endless loop]

Edit:
I wonder if it is related to port 164 (http://forum.doom9.org/showthread.php?p=1187583#post1187583) but checking my JMP, I get the following values:

U_Jimm = I & 0x3FFFFFF; // 25 bit constant
S_Jimm = (I & 0x2000000) ? ( (I & 0x3ffffff) | 0xFC000000) : (I & 0x3FFFFFF);

U_Jimm 0245AE89 S_Jimm FE45AE89
[dlx] 001EFC6C (I=2C): J (PC += FE45AE89): 0024AAF8 = 001EFC70 + -28987767

If Unsigned IMM is 0245AE89, then Signed Imm extended to 32bit becomes FE45AE89. Then if PC is 001EFC70, the out come is indeed: 0024AAF8.

Oopho2ei

19th October 2008, 11:11

Are there any saves of what the PC should be at each post_trap_mem milestone? My program runs on its on, and the registers all compare correctly (except R1 from the trap), it fixes R1, loads the next core and resumes.
You are supposed to use the traces of the program counter the instruction and the watchdog timer to verify that you are executing the right instructions in the correct order.

If Unsigned IMM is 0245AE89, then Signed Imm extended to 32bit becomes FE45AE89. Then if PC is 001EFC70, the out come is indeed: 0024AAF8.
Yes, sign extend the immediate value and then add the result to the pc and apply the mask (0x3FFFFC).

Accident

19th October 2008, 11:33

You are supposed to use the traces of the program counter the instruction and the watchdog timer to verify that you are executing the right instructions in the correct order.

Well, admittedly the Registers would only compare correctly to the saved files if the PC was, in fact, already correct. I was trying to find possible reasons as why it fails. So I guess it is running correctly. I just don't know why it would get stuck at trap 2029. But perhaps it sits there waiting for WD to expire or something, since I do nothing with WD. Should I decrease it every instruction or something similar?

Nor do I use INSTF.

Oopho2ei

19th October 2008, 12:06

I am glad to announce that the debugger seems to handle the new differential post-trap snapshot archives correctly. It is also considerably faster now: ;)

But perhaps it sits there waiting for WD to expire or something, since I do nothing with WD. Should I decrease it every instruction or something similar?
Yes, by the number of cycles the instruction needs. You can find the table in posting #25 (http://forum.doom9.org/showpost.php?p=1177651&postcount=25). You won't need it until after the first call of TRAP_Finished().

Nor do I use INSTF.
The instruction filter is set by cmd3A and you have to xor it with the instruction to get the decrypted instruction.

If you need help finding your bug write me a private message with the details (where did the pc desynchronize, differences between your memory and the snapshots, ...)

Accident

19th October 2008, 12:34

Yes, by the number of cycles the instruction needs. You can find the table in posting #25 (http://forum.doom9.org/showpost.php?p=1177651&postcount=25). You won't need it until after the first call of TRAP_Finished().

You know, I really should print the TRAP number:

[dlx] ** TRAP 00000010 (16) return: 0000103C
[bdtest] reached TRAP at 37236320:2676 PC=00001038.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002676.bin'
[bdtest] executed 37236320 instructions, 2677 traps (-1)

Which just happens to be TRAP_Finished(). heh oh dear...

The instruction filter is set by cmd3A and you have to xor it with the instruction to get the decrypted instruction.

Perfect, that is all I need to know to implement that, and WD.

Edit:

[dlx] ** TRAP 00000010 (16) return: 0000103C
[bdtest] reached TRAP at 37236320:2676 PC=00001038.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002676.bin'
[bdtest] break reached, PC=00001038: WD=FFFFFFFF
[bdtest] executed 1110978144 instructions, 2677 traps (2)

Ok, INSTF, WD and breaks are done. Should I go back and work on Traps, or carry on with Breaks?

Edit 2:

It isn't that I am ignoring your advice Oopho2ei, but Aes I understand, and is trivial to implement, where as I have no idea what a DeviceDiscover, or Slot, is so I am leaving those to later. Aes has been done.

[dlx] ** TRAP 00000110 (272) return PC=0001ABF0
[trap] trap_Aes(AES_DECRYPT_PLAYERKEYS): 0x2156dc->0x2156dc (3 key 4)
[bdtest] reached TRAP at 4290:5 PC=0001ABEC.
[dlx] comparing with core 'dat/post_trap_snapshots/post_trap_mem_000005.bin'
+VM.addr
-loadedcore
[dlx] compare complete. 0 differences.

Edit2:
Amusingly, MediaReadFile() should have taken 20 mins, but took all day. Looked like it was trying to read beyond the end of the memory, so I made a much more complicated read_file function. Turned out I hadn't got the right level of len ptr to ptr to ptr. Sigh.

Oopho2ei

19th October 2008, 13:49

Ok, INSTF, WD and breaks are done. Should I go back and work on Traps, or carry on with Breaks?
You should start working on the traps now. The group number of a trap is "trapID >> 8". I suggest you implement the groups in the following order: 2 -> 3 -> 4 -> 5 -> 1 -> 0 so you start with something easy like TRAP_XorBlock and advance to more difficult traps like the cryptography traps in group 1. You shouldn't start with events/instruction breaks because we don't fully understand this yet. If you think you can handle the snapshots pretty well you should consider to update your code to the new diffarchive snapshots (see posting #249 (http://forum.doom9.org/showpost.php?p=1197573&postcount=249) for details). The archives are much smaller and faster to work with but more difficult to implement.

Oopho2ei

19th October 2008, 23:00

I've made a package containing the latest debugger binaries, the snapshots of "The Day After Tomorrow" in the new much smaller diffarchive packages, the traces and all the other small files needed to run the content code.

all the files needed to run the machine and to study events: Full debugger package (http://uploaded.to/?id=50r7dx) (extracted: 672MB)

Simply extract this archive in a directory and run the debugger from the root of this directory (java -jar Debugger.jar). It should find all the files it needs automatically so there is no need to select anything from the menu. You only have to click run and watch the console output. The debugger stops on every execution break so the changes applied during the breaks to the virtual machine by the player can be examined. When you have studied the output press run again to continue until the next break. When a difference between snapshots/traces and the real state of our machine is detected a warning is sent to the console and the state of our machine is automatically adjusted. So don't worry. :)

You should consider to redirect the console output to a file (eg run with "java -jar Debugger.jar > output.txt") because it's quite a lot of data.

Edit: of course it's "mismatch" and not "missmatch". Sorry for that copy/paste typo.

Edit: TRAP_Schluppo is trap#0020. I gave this trap a (temporary) name so people (Schluppo and others...) stop using the former (wrong) name "TRAP_EventGet". I was told that "TRAP_EventGet" as described in the patent doesn't exist. TRAP_Schluppo might be the trap which passes the results of the content code to the player. The player then uses this data to repair sections of the audio/video stream.

Edit: if you rename the files/directories so the debugger can't find the traces/snapshots the debugger will run without guidance. This is only useful for developers to check where things start to get bad.

Oopho2ei

20th October 2008, 00:40

The obfuscation scheme guarding the player AES keys has been broken by one of our anonymous contributers. You can find the plain keys in posting #163 (http://forum.doom9.org/showpost.php?p=1187564&postcount=163).

This will surely ease implementation efforts. Great work!!! :thanks:

My obfuscated AES implementation is now obsolete ;)

Oopho2ei

20th October 2008, 15:36

It isn't that I am ignoring your advice Oopho2ei, but Aes I understand, and is trivial to implement, where as I have no idea what a DeviceDiscover, or Slot, is so I am leaving those to later. Aes has been done.
TRAP_Aes was difficult to implement until a few hours ago. Now you no longer need "my" obfuscated aes algorithm so it's much easier.

There has to be some kind of firewall around the virtual machine which validates all communication between the ( potentially hostile ) content code and the outside world (traps and events). Part of the rules which are enforced by that firewall are the parameter checks on every trap call. It's obvious that the traps and the event handling itself has to be carefully implemented. I believe this additional effort is necessary to prevent the content code from breaking out of it's sandboxed environment and do nasty things like gathering user information and "calling home" when it detects an unlicensed emulator. So because these additional security measures make things more difficult i suggested to test this code first with the easy traps. This (http://uploaded.to/?id=4nn0a3) is a draft of our debugger design (the interface contains the firewall).

Amusingly, MediaReadFile() should have taken 20 mins, but took all day. Looked like it was trying to read beyond the end of the memory, so I made a much more complicated read_file function. Turned out I hadn't got the right level of len ptr to ptr to ptr. Sigh.
Are you willing to provide us with your latest source code so we can have a look how it's going? Sadly you ignored my offer to use our assembla repository.

Oopho2ei

20th October 2008, 17:14

I have uploaded the two certificates we get from TRAP_DeviceDiscovery. You can view them here (http://svn.assembla.com/svn/bdplus/trunk/doc/player/certificates.txt). The currently unknown data section is supposed to contain:
- certificate version
- player manufacturer identifier
- serial number
- signature date and time
- device (player) attribute information
In the above order (source (http://uploaded.to/?id=1dn4ix) p.40 / figure 11). Maybe someone can identify pieces of the unknown data reading the corresponding text of the patent. :rolleyes:

Edit: These are the original certificates of the player. Nothing has been modified.

Accident

21st October 2008, 01:19

TRAP_Aes was difficult to implement until a few hours ago. Now you no longer need "my" obfuscated aes algorithm so it's much easier.

This is true, I simply call OpenSSL now, thanks to that. All of what I have done is based entirely on Your findings, I have yet had the chance to do any engineering of my own.

Are you willing to provide us with your latest source code so we can have a look how it's going? Sadly you ignored my offer to use our assembla repository.

Not ignored, but at the time, my code really didn't do any more than your sample so it felt pointless. But I am reaching a stage where it might lead somewhere, so I will register on assembla now.

Edit: Registration completed, same nick as here.

Oopho2ei

21st October 2008, 01:54

All of what I have done is based entirely on Your findings
Well i provide the raw data and other people analyze it together with me. If necessary they ask for more details (mostly via private messages). It's working pretty well and finally i am the one who presents the results. I surely did my part but it's only fair to say that other people who try to stay anonymous have contributed to this project a lot and i am thankful for it! :)

Not ignored, but at the time, my code really didn't do any more than your sample so it felt pointless. But I am reaching a stage where it might lead somewhere, so I will register on assembla now.
I have added you to the team. Welcome! :)
We only have 200MB disk space so try to avoid uploading binaries. You can commit your changes as often as you like.

schluppo

21st October 2008, 07:14

OK, just so that the obvious was stated in the thread: The new snapshots contain an additional (second) call of event 0x110(0,1) - followed by the usual bunch of trap-checking. After this event, the VM finally seems to be in a state to be willing to do the decrypting of the movie. Lots of calls of event 0x220(0,1,n) follow. And for each invocation of event 0x220, the VM just seems to test different traps (as noted in postings #211 (http://forum.doom9.org/showthread.php?p=1193136#post1193136) and #213 (http://forum.doom9.org/showthread.php?p=1193182#post1193182) - note that I made the experiments from these two posts without snapshot-guidance and still the results were quite correct). After the trap-testing, each event 0x220 is treated with a TRAP-SlotAttach(0,14) (I don't know whether successful or not) and a TRAP-SlotRead(0,*) afterwards. However, the data which is read from slot 0 is always the same, so this is certainly no 'input-channel'. Two calls of TRAP_DeviceDiscovery get the two certificates before the final TRAP_Finished ends the treatment of the current event.

Furthermore, all the interruptions just change the event-id and event-parameters and nothing besides, so that the first 0x1000 bytes of the VM's memory are no 'input-channel' either. Also I observed that (when restricting attention to the first 0x1000 bytes) the VM is not changing more than the first 0x3F bytes during treatment of event 0x220, meaning that also the output needs to happen in some other way (64 bytes is surely not enough output for 30 seconds or so of stream-repairing).

Currently I am guessing, that all the traps that are being executed after event 0x220 was invoked are just concerned with verifying that the VM is still running unmodified. I further guess that the VM is not using traps to communicate the important data with the player. One of the patents mentions a conversion table, so maybe the parameter <n> of event 0x220 is just used to copy a specific part of this table to a predefined area inside the VM's memory and then the player can later access this table on it's own and replace the bad data in the video stream with the appriopriate entry in the currently 'active' part of the conversion table. This would mean, that there is apart from parameter <n>, no real data-input to the important event.

An another note, there are still bugs left in the treatment of at least TRAP_DeviceDiscovery, TRAP_AES and TRAP_MemSearch. More information about that soon. Also TRAP_DiscoveryRAM is still problematic (seems like parts of the RAM change over time after all :/)

Edit: The details about the bugs:

#004433 TRAP_DeviceDiscovery (00000001, 00000001, 003FFD48, 001EF810);
[!] register missmatch after trap call: r01: 00000000 (vm) != 80000001 (snapshot)

Bad endianness:
#004006 TRAP_DeviceDiscovery (00000002, 00000000, 001EE854, 001EE7D0);
[!] memory missmatch after trap call: 1EE854: 52 (vm) != 00 (snapshot)
[!] memory missmatch after trap call: 1EE857: 00 (vm) != 52 (snapshot)
[!] memory missmatch after trap call: 1EE858: 38 (vm) != 07 (snapshot)
[!] memory missmatch after trap call: 1EE859: 04 (vm) != 80 (snapshot)
[!] memory missmatch after trap call: 1EE85A: 80 (vm) != 04 (snapshot)
[!] memory missmatch after trap call: 1EE85B: 07 (vm) != 38 (snapshot)
[!] memory missmatch after trap call: 1EE860: 04 (vm) != 00 (snapshot)
[!] memory missmatch after trap call: 1EE863: 00 (vm) != 04 (snapshot)
[!] memory missmatch after trap call: 1EE864: 01 (vm) != 00 (snapshot)
[!] memory missmatch after trap call: 1EE867: 00 (vm) != 01 (snapshot)

#008136 TRAP_MemSearch (001CFA58, 00000014, 001CFA6A, 00000002, 001CF998);
[!] memory missmatch after trap call: 1CF999: 00 (vm) != 1C (snapshot)
[!] memory missmatch after trap call: 1CF99A: 00 (vm) != FA (snapshot)
[!] memory missmatch after trap call: 1CF99B: 00 (vm) != 6A (snapshot)

#007861 TRAP_Aes (001EE210, 001EE220, 00000000, 001EE1C0, FFF10000);
[!] register missmatch after trap call: r01: 00000000 (vm) != 80FFFFFF (snapshot)

Edit2: If you'd like to 'cheat', it would probably be possible to compare a stream-file for this movie with just AACS removed but BD+ intact and then the output of AnyDVD for this stream-file. The first 20 or so sets of 5 different bytes (such as described in posting #13 (http://forum.doom9.org/showthread.php?p=1174977#post1174977)) should appear somewhere in our VM's memory after treatment of event 0x220 (0,1,1) - telling us the location over which the communication is happening. But of course doing it this way would be no fun :rolleyes:

Edit3: Just a wild idea: Could this trap here:
#002677 TRAP_Schluppo (00000000, 00001000);
Be something like TRAP_SetCommunicationArea(len,pArea)?

Edit4: I found that after (and before) execution of
#007426 TRAP_Schluppo (000FCD9E, 000EFAF8);, there is a big structure at VM-memory 0x000EFAF8, to me it looks like the conversion table. The structure is something like:

<header>
then lots of blocks, where a block looks like this:
32 bit = <blocklen>
<blocklen> * 32-bit address (?) - increasing all the time
~1,5 * <blocklen> bytes - correct data for the according address?

At this point (trap #7426), the values (addresses?) in the structure go up to 0x9EB01D4 before the structure ends. In order to address the whole big stream-file, 32bit wouldn't be enough, so could it be, that the table is holding 'sector numbers' instead of byte-addresses? Structure length is quite close to 0x000FCD9E (that's almost one megabyte).

Accident

21st October 2008, 14:09

If I may ask, what should the VM do when WD expires (we get a break?) I've been perusing the thread, but I am still not certain. Some java mention "events" that perform:

vm->R[28] = vm->PC;
vm->PC = 0x1000;
vm->WD = 0x7FFFFFFF;

But the following Trap differs vastly. (Perhaps I should finally read a post_break snapshot.)

Oopho2ei

21st October 2008, 14:58

If I may ask, what should the VM do when WD expires (we get a break?) I've been perusing the thread, but I am still not certain.
You are not certain because we aren't certain either. :)
This topic is still under research so you shouldn't start to implement this yet. What we currently think happens is that when the content code calls TRAP_Finished it signals to the player that the virtual machine has finished all tasks and is now idle. The watchdog timer is now set to 4000 which causes the "idle loop" of the content code to be interrupted every 4000 cycles. On every interruption the player checks an event buffer ("mailbox") for new events. If there is an event then the vm memory section 0x00-0x40 is modified and the registers are changed as you described. If there is no event then only the watchdog is reset to 4000 and execution continues in the idle loop. A watchdog timer value of 0x7FFFFFFF basically means "don't interrupt the execution" whereas a value of 4000 means "do event polling".
Every call of TRAP_Finished restores the program counter from register 28 and sets the watchdog timer to 4000.

Edit2: If you'd like to 'cheat', it would probably be possible to compare a stream-file for this movie with just AACS removed but BD+ intact and then the output of AnyDVD for this stream-file. The first 20 or so sets of 5 different bytes (such as described in posting #13 (http://forum.doom9.org/showthread.php?p=1174977#post1174977)) should appear somewhere in our VM's memory after treatment of event 0x220 (0,1,1) - telling us the location over which the communication is happening.
I like the idea. Thanks for the bug reports. I am currently trying to fix them. We probably need the algorithm which creates the "free space pattern" for TRAP_DiscoveryRAM. Also please consider to upgrade to revision 96 which is much faster with snapshots.

There has been an "Accident" with our repository which i could repair with the backup of yesterday evening so you need to checkout the entire repository again if you updated today because revision 97-99 (currently) don't exist.

Oopho2ei

21st October 2008, 19:29

Some details about TRAP_DiscoveryRAM and the free space pattern:
cd f0 a2 e2 ed 88 e3 98 bc 66 c1 7c f4 db 47 96
f6 6c 5e 11 af 2f 40 42 41 07 de 4c 8a 63 4d 51
c0 9b 38 27 19 03 97 65 3d 44 ac a7 18 a0 61 13
b3 b4 c6 21 15 e0 c5 0f 78 c4 ef 2c 53 26 2e 67
54 5f d5 c8 aa 17 46 95 a3 94 fb ba d3 bd 64 2a
bf 34 48 35 43 d7 f5 cf 90 92 2d b5 5d 93 99 50
74 72 31 04 58 10 5a 7f ff ca 55 37 b2 dd e5 0a
0d 69 b1 3a 00 3c ea 22 32 8d f2 9c 86 1c b0 76
30 01 d2 06 bb 77 f3 80 e8 a6 05 ec 89 49 fd d9
d6 d4 45 6f 4f b8 33 57 d8 87 a8 9e f9 5c 23 b6
6b eb 7e 1f 02 fe 85 e9 12 c9 ae 08 9f 52 25 71
09 3f 29 68 3b 1a e7 91 59 7a 6e 8e 56 a4 1d 1e
a9 9a df 70 8c 4e 4b dc ee 36 8f c3 83 82 e4 8b
79 6d 0c a1 7d 39 4a be fa ab d1 c7 28 7b cc f7
b7 ad 62 b9 d0 f8 f1 0e 1b cb da e6 2b 60 16 c2
81 0b 73 a5 20 84 5b 24 9d 75 e1 ce 14 6a 3e fc
That is the lookup table (called FREESPACE_PATTERN_LUT) needed and here is a sample implementation which is currently used in the debugger:
private byte[] BuildVirtualPlayerMemorySection( int src, int len )
{
byte[] buffer = new byte[len];
int address;

// fill the section with free space pattern
for (int i = 0; i < len; i++) {
address = ( src + i ) & 0xFF;
buffer[i] = constants.FREESPACE_PATTERN_LUT[ ( address * address ) & 0xFF ];
}

// write real player data if there is any
for ( int i = 0; i < len; i++ ) {

address = src + i;

if ( address >= 0x00000000 && address <= 0x000001FF ) // 0x00000000-0x000001FF : player name
buffer[i] = playerName[ address ];

if ( address >= 0x00000200 && address <= 0x0000021F ) // 0x00000200-0x0000021F: player version
buffer[i] = playerVersion[ address - 0x200 ];

if ( address >= 0x00000220 && address <= 0x0000031F ) // 0x00000220-0x0000031F: unknown player info
buffer[i] = playerInfo[ address - 0x220 ];

// TODO: there is more
}

return buffer;
}
The variable <src> is the virtual memory address where we have to read the <len> bytes of data from.

Some parts of the virtual player memory seems to contain real data at fixed addresses. The (undefined) rest of it is filled with this pattern. I still don't know if this pattern is just garbage data or required by the specification. You should be able to find the answer when you set data breakpoints on this pattern and run the debugger. :)

Schluppo: Could you also look at TRAP_DiscoveryRAM in revision 97 (and later)? As you can see the RAM.bin is no longer used instead everything is filled with this pattern and later partly overwritten with contents we know (see above). You will see from the console output which calls of TRAP_DiscoveryRAM don't return the pattern. The first difference is at this call:
#002151 TRAP_DiscoveryRAM (0026D976, 001EFC94, 00000080);
Also the events.java is yours for your experiments. I won't touch it. :)

Edit: This (http://uploaded.to/?id=a7kknn) is the latest binary of the debugger. Simply replace the "Debugger.jar" of the package from posting #256 (http://forum.doom9.org/showpost.php?p=1197926&postcount=256) with this one. It's faster but the changes in the implementation of TRAP_DiscoveryRAM causes a lot of warnings to be sent to the console now. You may delete the RAM.bin and the crypto directory because these files are no longer needed.

Accident

22nd October 2008, 00:50

You are not certain because we aren't certain either. :)
This topic is still under research so you shouldn't start to implement this yet. What we currently think happens is that when the content code calls TRAP_Finished it signals to the player that the virtual machine has finished all tasks and is now idle. The watchdog timer is now set to 4000 which causes the "idle loop" of the content code to be interrupted every 4000 cycles. On every interruption the player checks an event buffer ("mailbox") for new events. If there is an event then the vm memory section 0x00-0x40 is modified and the registers are changed as you described. If there is no event then only the watchdog is reset to 4000 and execution continues in the idle loop. A watchdog timer value of 0x7FFFFFFF basically means "don't interrupt the execution" whereas a value of 4000 means "do event polling".
Every call of TRAP_Finished restores the program counter from register 28 and sets the watchdog timer to 4000.

That is quite interesting. So something of the lines of:
trap_finished(): WD=0x4000, PC=R28 (even the first time?)
VM runs endless loop, until WD expires
R28=PC, WD=4000, PC=?
instructions executes
trap_finished() called again.. WD=0x4000, PC=R28... ?
-----

Either way, I will return to filling in the TRAP code for now, I have quite a few to go. I will also glance at the diffarchives, since waiting 2600 traps takes a while.
----
Edit: I can't get any post_break_mem snapshots to work, probably as I don't know what PC to set, I have implemented all the Traps that make sense to me. But I am still missing PrivateKey, *Slot (wtf is a slot?) traps. Not entirely sure what to do next.

Oopho2ei

22nd October 2008, 09:24

That is quite interesting. So something of the lines of:
trap_finished(): WD=0x4000, PC=R28 (even the first time?)
Yes, that seems to be the case.

VM runs endless loop, until WD expires
R28=PC, WD=4000, PC=?
If the "mailbox" is empty then nothing happens. Execution continues without any changes. The interesting case is when really an event occurred so the "mailbox" is not empty. Then the PC is so set to 0x1000 (always?) and r28 and WD are set as you described.

instructions executes
trap_finished() called again.. WD=0x4000, PC=R28... ?
The watchdog is set to 0x7FFFFFFF so that the virtual machine won't be interrupted until it has finished processing the current event (signaled by the call of TRAP_Finished) regardless of how many more events are in the mailbox.

Either way, I will return to filling in the TRAP code for now, I have quite a few to go.
Actually Schluppo was so eager to study the events that he kept asking me for new snapshots. He has made only very few postings lately so it seems it's more complicated than we thought or maybe he is no longer idle.

I will also glance at the diffarchives, since waiting 2600 traps takes a while.
The DAT v1.03 package contains 10929 post-trap snapshots and 1816 post-break snapshots which altogether need less than 11MB disc space. :)

Oopho2ei

22nd October 2008, 18:26

We have identified the remaining address ranges of the virtual player memory which contain real data. You can view the updates here (http://svn.assembla.com/svn/bdplus/trunk/doc/player/memory_layout.txt). It looks like most of the data is changing over time.

Edit: the different pattern in section 00250000 - 0028FFFF is created like this: if ( address >= 0x00250000 && address <= 0x0028FFFF ) // 00250000 - 0028FFFF : pattern similar to freespace pattern
buffer[i] ^= (int) ( ( 3 * address * address + 1 ) & 0xFF );

Edit: this could be a pattern which indicates that there should be real data here but this particular player doesn't provide the data.

schluppo

23rd October 2008, 05:13

TRAP_DiscoveryRAM is giving lots of problems now (seems the implementation is not finished yet). Any idea how to treat the areas of memory that are changing over time?

Event handling is actually quite easy - I think, there is the following standard program that all the players try to invoke on the BD+ -VM:

intialize + auto-test VM
event 0x0110 (0,0xFFFF) (reset conversion-table)
event 0x0210 (0,1) (check whether player is ready for playback)
event 0x0110 (0,1) (test VM and load conversion table)
event 0x0110 (0,1) (test VM and load conversion table)
event 0x0220 (0,1,1) (partially test VM)
event 0x0220 (0,1,2) (partially test VM)
event 0x0220 (0,1,3) (partially test VM)
...
event 0x0220 (0,1,n) (partially test VM)
event 0x0010 (shut down)

If any of the VM-tests fail at any time, the conversion table is destroyed and VM execution is halted.

However, since our VM is doing all the work (once it finally works 100% correct), we can just invoke the events in this order and get the data needed for decoding the movie from the conversion table. I think, that the non-event interruptions can be omited altogether (at least for the first 4 "intializing" events), since we don't have to wait for user-(or player-)input when we just want to decrypt the movie.

Anyway, I am getting the impression, that the player is not just taking the data from the conversion table and then putting it to the right place in the stream. I rather think, that there is some algorithm being executed on the data from the conversion table before the "correct" data for the stream is obtained. I found out, that for every entry in the conversion table, there are 16 bytes assigned to it (AES-key, anyone?)

If you have access to the player's routines, it would be helpful to know, what the player is doing with the data that it is (hopefully?) reading from the conversion table (the player should read from the conversion table whenever it encounters 5 "bad" bytes in the stream. The conversion table for "DATv1.03" is laying in VM-memory at 0xEFAF8 after finished treatment of event #4).
Also it is still necessary to find out the connection between the addressing-value in the conversion table and the actual position of the "bad" location on the disk / inside a file (the first entry in the conversion table is holding address-value 0x0009090A and assigns the following 16 bytes to it: 0xB6BE742B 0x32C794D0 0xBCCCB272 0x76B8AF89 - to what is 0x0009090A referring and how are the 16 bytes used to get correct data for this block?).

Finally, here is some more info on the conversion table found in "DATv1.03" (correcting some faults from my last posting):

Start-Offset: 0x000EFAF8.

0x000EFAF8: 00 00 24 00 53 01 01 00 // unknown - table-length?
Then 0x0153 32-bit adresses. // block-adresses?
0x000F004C: 00 00 00 00 // unknown

0x000F0050: 82 00 00 00 // block 1 length
Then 0x0082 32-bit adresses. // block 1 row 1
Then 0x0082 times 16 bytes. // block 1 row 2

0x000F0A7C: D0 00 00 00 // block 2 length
Then 0x00D0 32-bit adresses. // block 2 row 1
Then 0x00D0 times 16 bytes. // block 2 row 2

etc...

Then when all blocks are finished:

0x001EC424: 02 00 00 00 0F 00 34 00
Then 0x0034 32-bit values in the following pattern (n = 0..C):
0F 00 n2 CA 0F 00 n6 CA 0F 00 nA CA 0F 00 nE CA

The table seems to end at 0x0001EC4FB

Edit: If this should really be the complete conversion table, giving for each adress an AES-key that can be used for decoding the according part of the stream, then it suffices to just invoke the first 4 events and then copy the complete conversion table (TRAP_0x20 will tell us for every title where the table can be found in VM-memory). A different program / function could then go through the .m2ts files and decode all the bad blocks (using the data from the table).

Edit2: Just an observation: When using snapshots and outputting all the trap-logs etc., the debugger's VM is running around 5 times slower on my computer, than the VM on the player that was used to create the snapshots. I hope this is not creating any timing problems (BD+ code could be expecting certain things to be finished in a specific time-window or something similar).

bourke

23rd October 2008, 10:05

Edit2: Just an observation: When using snapshots and outputting all the trap-logs etc., the debugger's VM is running around 5 times slower on my computer, than the VM on the player that was used to create the snapshots. I hope this is not creating any timing problems (BD+ code could be expecting certain things to be finished in a specific time-window or something similar).
Can't we slow the system clock down at a driver level?

Oopho2ei

23rd October 2008, 12:38

TRAP_DiscoveryRAM is giving lots of problems now (seems the implementation is not finished yet). Any idea how to treat the areas of memory that are changing over time?
I have just committed the update you can find in posting #268 which reduces the number of warnings. Either build from svn or look at the bottom of this posting to find a link to the latest binary.

If you have access to the player's routines, it would be helpful to know, what the player is doing with the data that it is (hopefully?) reading from the conversion table (the player should read from the conversion table whenever it encounters 5 "bad" bytes in the stream.
When calling TRAP_Schluppo(P0, P1) the player is reading VM memory from both P0 and P1 using the other parameter as length in each case. So it first reads P0 bytes from P1 and then P1 bytes from P0. :confused:
Yes i have seen how the table (which is stored at the beginning of 00002.svm) is parsed and i also saw those 16 bytes blocks being decrypted by xor with some other values which possibly come from the VM memory too. It would make sense if we need other data from the content code to work with this conversion table because otherwise we wouldn't need to execute the content code.

Edit2: Just an observation: When using snapshots and outputting all the trap-logs etc., the debugger's VM is running around 5 times slower on my computer, than the VM on the player that was used to create the snapshots. I hope this is not creating any timing problems (BD+ code could be expecting certain things to be finished in a specific time-window or something similar).
Speed is only a problem if we want real time decoding during playback. It should be fine to first repair/decode the files on disc. It runs pretty fast if you rename the trace and snapshot directory so the debugger runs free. You can use the new binary (http://uploaded.to/?id=j44ic4) or build from the repository to verify that it also calls TRAP_Schluppo correctly without snapshots/traces.

Edit: The content code seem to modify these 16 byte blocks after loading it from 00002.svm#005051 TRAP_MediaReadFile (00098DEC, 00000000, 00000000, 000EFAF4, 000EFAF8); // read 0x000EFAF4 bytes from section 0 of 00002.svm and store them at 0x000EFAF8
...
// decrypt table contents?
...
#005052 TRAP_Schluppo (000FCD9E, 000EFAF8); // send result (0x000FCD9E bytes at 0x000EFAF8) to player
When the player gets the data it parses the table and itself modifies the encrypted sections (the section following the index section) by xoring every of the 16 byte block with some unknown value. For the first block with length 0x82 this value is "AB EE B5 BA C2 E8 D8 18 FF ED FF 8B 85 D0 5E 8C". When this is done i can see a regular pattern in this section so i doubt these are keys. For example the second byte of every resulting 16 byte block is often repeated in the next block. I couldn't find any further references to the resulting decrypted table during the trap execution so it it stays somewhere in memory and is probably used later during movie playback.

Edit: That's the final table which is kept in player memory during playback: conversion table? (http://uploaded.to/?id=ex4tl5)

Edit: Another note on performance. The C implementation of Accident will probably be much faster so i don't think this will be an issue.

Oopho2ei

24th October 2008, 19:29

I have made some progress but it's more difficult than simply coping parts of the conversion table over the broken parts of the media content. So please be patient. :)

Oopho2ei

25th October 2008, 17:15

To give you an idea of what is coming: content repair code (http://uploaded.to/?id=lom2ej).

That's the code which does the actual replacement of the media stream based on some data structures created from the conversion table entry: "1d 50 c1 91 f0 2f 4c c8 43 21 5d f9 f3 68 f1 05".
The first array 'A' is a constant (for one bad spot in the media stream) whereas the second array 'B' is different each time and so are the shift lengths which itself are calculated from 'v0'. So this basically means 'B' is actually an array of 6 bytes with each entry expanded from 8 bytes to 16 bit by placing the 8 bit at a random position in a random 16 bit value. This looks like obfuscation to me. Will all the obfuscation removed this code probably collapses to a simple xor loop of two 5 byte arrays :)

Btw. If you look at byte position 6 and 7 of the conversion table entry (see above) you will find 0x4C and 0xC8 which also occur at the end of the array 'A' which is no accident.

Edit: Have a look at posting #13 (http://forum.doom9.org/showpost.php?p=1174977&postcount=13) for more details.

bshep

25th October 2008, 17:57

To give you an idea of what is coming: content repair code (http://uploaded.to/?id=lom2ej).

On line 16 the value assigned to v0 is never used, later it is overritten on line 22 which is the value that is used in the future.

Also B[0] doesnt seem to be used for anything.

So it seems that the data in A[] is the corrupted data somehow modified and B[] is the 'key' information obfuscated with some random data for which v0 is the 'seed'. I agree that once the obfuscation algorithm is understood this can probably be collapsed to a simple XOR.

Oopho2ei

25th October 2008, 18:55

Just to confirm everyones guess about the origin of A:
<---- ????? ----> <---- A ----> <---- ?? ---->
E = 1d 50 c1 91 f0 2f 4c c8 43 21 5d f9 f3 68 f1 05
A = 21 43 5D C8 4C

A[0x00] = E[0x09] = 0x21
A[0x01] = E[0x08] = 0x43
A[0x02] = E[0x0A] = 0x5D
A[0x03] = E[0x07] = 0xC8
A[0x04] = E[0x06] = 0x4C
The permutation of A is probably part of the obfuscation.
Note that E was already transformed by the player when every entry of the table (sent by the content code via TRAP_Schluppo) was xored with "AB EE B5 BA C2 E8 D8 18 FF ED FF 8B 85 D0 5E 8C". Also the first 6 bytes of E (as above) are transformed to "4A 1A A0 53 64 B1" this is why i believe E consists of at least 3 parts.

@bshep: don't worry about v0 and B[0]. Your guess about A and B may be right.

Oopho2ei

25th October 2008, 21:09

One of our anonymous contributers figured out that the certificates are signed with a 1280bit RSA signature and also found the public key of the license administration.
the public key is (n,e) with e = 3 and n =
8B169F529C28B5D45DB5D1607B831BED31381D38AEF561A43E744326DD00765E
E7A47F353D4A8C507752B08A6671259AAF140E86EEB1D05D344EF801A5AFB150
3A82BE089DCF25618852199D26CC79AE99466A231999AAC6C26E7DDA662304A7
72D1B304C9CD0C724434D640E29BE64FBBE1E7993A30939D6FB925AE0C350896
14F89FBAE9B931FC01D4D10732EB62CA8878E1894BD82F3007806D75CE172B57

To give you an example how to check the signature of a certificate (http://svn.assembla.com/svn/bdplus/trunk/doc/player/certificates.txt):
signature from certificate 1:
03B04675B5AAB8B1E66974F1C90667C846C7CDF5259F9E95139BF12D6A46312F
7575E28404ABA98914EEE83A65F752EFCFA115D1145BC605AAFD68D9287D4F2F
99536F014ED21717871B2B7CF3299499BBBA8AF5D459AEE240A314D51AE4F991
D445585C14E291041C4358B337CF9F14E48699DF291A5CB571CB61FD68ACD7E6
1EC2FD1296884E0F95683760929B8EB34009C69974E839E2E4B63F75FAF3726F

SHA-1 Hash of certificate 1:
77A325225B91CEF17DCD6F1785016A9FFFFD4C8E

result of encrypting the signature with the public key:
0001FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00302130
0906052B0E03021A0500041477A325225B91CEF17DCD6F1785016A9FFFFD4C8E
You can verify that the last 20 bytes indeed match our calculated SHA-1 hash. Therefor the certificate is valid.

Great work again. :thanks:

PS: if you don't know how to encrypt a message with RSA it is simply: result = message ^ e mod n

Oopho2ei

25th October 2008, 21:47

I was given a OCR pdf version (http://uploaded.to/?id=56kmvi) of patent US 2007/0033419 Al. This might be helpful if you want to search the document and see the pictures.

Oopho2ei

26th October 2008, 02:21

I have looked at the code which creates 'B' and i saw that "AB EE B5 BA C2 E8 D8 18 FF ED FF 8B 85 D0 5E 8C" which is initially used to transform every entry of the conversion table (fix-up table) is used here too. :confused:
This is something i did not expect and it might be that this xoring with "AB EE B5 ..." is not part of the preparations to use the conversion table. Instead it might be an attempt to make it more difficult to steal this precious table from the players memory using various memory dumping techniques. It surely is an easy target because the conversion table has to be present in memory during playback. So my suggestion is to continue working with the table as it passed from the content code to the player with TRAP_Schluppo.

One more thing: It looks like 'B' only changes around every 20-30s (telling from the memory write accesses over a certain period of time). I could also confirm this experimentally. If i change B i get a lot of distortions (video freeze over several second) whereas if i change A only a few frames look bad.

Edit: There might also be a relationship between the number of entries in a block of the conversion table (eg. first block has 0x82 entries) and the number of "bad spots" in one video segment.

cyt0plas

26th October 2008, 07:39

i make daily backups of TRAC and SNV so we can quickly move if necessary. But reverse engineering for interoperability (bd+ for linux) is normally considered legal so i doubt there will be any problems. :)

If you would like, I have a SSL-Protected SVN server (with access control) that could be used. I own the box personally, and have a good relationship with the ISP - in the event of a DMCA takedown notice, or similar, I would have an opportunity to take advantage of the counter-takedown provisions of the DMCA. Many ISPs do not provide notice, and simply shut down the box.

Oopho2ei

26th October 2008, 12:21

Thanks. I am thinking about a new program which repairs all the "bad spots" in the media stream using the conversion table and which i won't upload to our repository. The debugger can be used to generate this table but apart from this it is only useful to study the content code so i hope it is legal and doesn't infringe any patents. I am also hoping that this project doesn't draw any attention in the media because we don't do any better than commercial tools like AnyDVD-HD except that we support platforms other than Microsoft WIndows.

Oopho2ei

26th October 2008, 14:45

Some success. :)
These are the first entries of the plain conversion table as it comes from the vm memory:
<---- ????? ----> <---- A ----> <---- A' ---->
b6 be 74 2b 32 c7 94 d0 bc cc b2 72 76 b8 af 89
73 84 14 69 fc ee 88 0a 8a ae 33 41 f8 50 02 ad
ac 84 f4 00 e2 47 38 0e 56 3e a1 49 0b 82 39 21
a2 84 04 1e 5c 35 de f6 6a e1 0f bc 92 5d 3a cd
I have calculated the corresponding B and B' for the first segment:
B = 3EFDFC42F8
B' = 3F99F3C46C
You can verify that:
0x94d0bcccb2 XOR B = 0xAA2D408E4A
0x7276b8af89 XOR B' = 0x4DEF4B6BE5
0x880a8aae33 XOR B = 0xB6F776ECCB
0x41f85002ad XOR B' = 0x7E61A3C6C1
...
Each of the 5 bytes overwrite a "bad spot" as you can see from posting #13 (http://forum.doom9.org/showpost.php?p=1174977&postcount=13).

Oopho2ei

27th October 2008, 00:52

Every entry of the first block is xored with the value FC A4 D4 78 56 76 3E FD FC 42 F8 3F 99 F3 C4 6C
which can also be found in the player memory. I am currently tracing it to it's source. Anyway the resulting decrypted first block is: <--- header ---> <- piece 0 -> <- piece 1 ->
4A 1A A0 53 64 B1 AA 2D 40 8E 4A 4D EF 4B 6B E5
8F 20 C0 11 AA 98 B6 F7 76 EC CB 7E 61 A3 C6 C1
50 20 20 78 B4 31 06 F3 AA 7C 59 76 92 71 FD 4D
5E 20 D0 66 0A 43 E0 0B 96 A3 F7 83 0B AE FE A1
B3 1E D0 65 8B 08 02 47 5D 56 F6 55 D3 9A 55 8D
51 26 10 19 3E AB E7 8C 57 D3 80 CF 51 4C 72 4E
BD 11 20 E8 0B 62 03 CF C3 C5 D9 32 7F 0F 21 6F
7F 0D 01 13 B5 B4 EB 25 EC 62 A7 4C AC 61 66 27
7A 05 50 14 77 28 FE 1B B7 DA 6C 5E 1C 7A 2E 69
96 02 10 41 96 2F D1 3E 88 66 BA 01 84 2E 9F 9C
A9 00 B0 3E 59 93 56 18 1A 8B D8 10 00 02 5C 00
AC 08 B0 2B A1 80 A4 30 D1 66 6E AC 17 12 1D 3B
BF 04 10 25 11 5F 60 F9 34 FA 12 4F A7 0E 06 2E
92 02 A0 27 54 14 2D B1 22 AA 27 9B 2A 56 C6 CF
4B 05 40 0D 27 A3 15 75 17 1D B4 46 8F FF 7C B6
75 02 10 26 68 AC AD 05 9A 4B 0E 78 9A 4D BD 4A
AB 06 60 1B AE 4F 4D 33 12 1B 57 17 05 88 B3 71
BF 06 F0 19 94 5B 4F 64 CF 63 4A F7 28 69 62 9A
63 06 60 26 7C 1F B4 45 71 B3 61 09 72 99 D2 A4
A9 04 B0 26 70 0D EB B9 1C A4 D4 09 02 A9 5A 77
53 04 A0 1B 58 15 2B AD E5 64 84 B2 F5 9D 16 DE
40 01 40 17 A1 7E 09 E0 31 AE 1C CA 7C 9C D3 8B
68 06 F0 44 92 9C B6 6B 1A 67 50 CC 59 9E 7D D4
9A 01 20 39 A3 B3 69 9C DF 4E 8F 0A F0 BB 1C 3C
69 04 80 23 2D 4F 0F F6 F7 C6 11 8D 47 0B 49 20
4B 06 D0 0F 5A 57 D8 55 A0 63 EB DA EE AB DF 70
9B 07 90 0E 14 98 C1 29 B3 62 FC 8F F4 D4 BA BF
AB 07 C0 23 7D 35 54 24 38 CA AE 2E 0F 6F 9B 15
8A 05 00 5C A7 73 CA 62 D3 28 CF 8B AD 23 AF 84
68 15 50 1B A6 54 9A D8 6A 98 0D 82 4B 4F 53 6C
79 04 20 35 B1 AD 81 A8 F8 A0 53 15 AD 88 65 54
65 07 50 28 A0 70 69 B7 57 54 C1 F4 25 2C 4F 12
Every row contains the information to fix two broken spots in the media file. The replacement data which overwrites the broken spots is called piece 0 and piece 1 here.
Notice that the last two bytes of the header are the address of the bad spots in the plain TS buffer (size: 0xC0 bytes) for piece 0 and piece 1 respectively. For example the first change occurs at address 0x072D4764 (as you can see in posting #13 (http://forum.doom9.org/showpost.php?p=1174977&postcount=13)) so the data is written in the buffer starting at address 0x072D4764 mod 0xC0 = 0x64. The other four bytes of the header probably contain the rest of the address to determine the exact position of the change in the file "00001.m2ts". The first two bits of the header have a special meaning. According to the checks in the code they can either be '01' or '10'.

This file (http://uploaded.to/?id=6m5xrw) contains a comparison of the outputs of anydvdhd and dumphd for "The Day After Tomorrow". It expands your view beyond what is shown in posting #13 (http://forum.doom9.org/showpost.php?p=1174977&postcount=13). It is not a complete comparison because the file would have been too large so i stopped after a few minutes. The broken spots of the movie are scattered randomly over the entire file with large spaces in between relative to their size so the contents of this file doesn't contain a single frame of the movie or anything you could playback.

Oopho2ei

27th October 2008, 18:53

Has anyone searched the vm memory for the following segment keys?
FC A4 D4 78 56 76 3E FD FC 42 F8 3F 99 F3 C4 6C (first segment)
0E 7B 1F D9 95 E0 37 EE 7E 96 C5 61 63 76 3E FA (second segment)
48 69 0D 2D 9A 41 BE 2E 82 52 64 88 B1 E7 31 9F (third segment)
AA AB 5B E9 44 CC 38 B3 93 21 6D 1C C5 AD E0 DC (fourth segment)
...
Maybe after every event #220 the segment key (corresponding to the segment number passed as third parameter) is stored at 0x000010 in the virtual machine memory for the player to fetch?

You have no idea how often a segment key is copied from one module to another until it is finally applied in my player. I am glad i don't have to do this again :)

Edit: the Debugger doesn't give the correct value so something is wrong. :rolleyes:

Edit: I have checked my player again and after Event #220(0,1,1) i can find FC A4 D4... in the vm memory at address 0x10 before TRAP_Finished is called. Even if i run the debugger with snapshot/trace guidance i get a wrong value at 0x10. :confused:

Edit: forget it. The debugger is giving the correct result when running with snapshots/trace guidance:
#007599 TRAP_DeviceDiscovery (00000001, 00000001, 00022B48, 001EFA58);
#007600 TRAP_DeviceDiscovery (00000001, 00000002, 00022B48, 001EFA58);
#007601 TRAP_Finished ();
FC A4 D4 78 56 76 3E FD FC 42 F8 3F 99 F3 C4 6C
#000206 Interruption
#000207 Interruption

But sadly without the help of the snapshots and traces the result is wrong:
#007567 TRAP_DeviceDiscovery (00000001, 00000001, 00022B48, 001EFA58);
#007568 TRAP_DeviceDiscovery (00000001, 00000002, 00022B48, 001EFA58);
#007569 TRAP_Finished ();
60 FC A5 8B 89 59 38 B4 29 F0 9F 81 8C EB B6 EC
#000206 Interruption
#000207 Interruption
I guess this means the content code has detected an implementation error. The problem could be the unfinished implementation of TRAP_DiscoveryRAM like Schluppo suggested. Also note that fewer traps have been executed in this case.

Edit: Schluppo seems to be right. When i change the contents of the virtual player memory the section key is changing too. Therefor i will pay attention to TRAP_DiscoveryRAM now.

Oopho2ei

28th October 2008, 03:18

It looks like some values needed for the calculation of the segment keys are stored in slot 0 or at least the contents of slot 0 influences the result. Anyway there was a stupid bug in TRAP_SlotWrite (the slot payload was copied from the same address as the hash :rolleyes:) which caused the calculation to fail. It seems the conversion table and the segment keys (used to decrypt the entries of this table) are correctly calculated now without the help of snapshots or traces.

I've added a switch 'autoEvents' to the event class which determines if an event is generated every time the calculation is interrupted or if the debugger should issue events like the player in the snapshots. The default is to generate events automatically now.

Schluppo made a patch which writes the conversion table into a file "conv_tab.bin". You will get the segment keys one by one starting with event #000004 by pressing F5 or clicking the "run" button. You can compare them with the keys from the previous posting.

That's a snapshot of the debugger with all files needed: Debugger (testing) (http://uploaded.to/?id=6f1ph8)

Now all that's left to do is decrypting the entries of the conversion table with the segment keys (simple xor) and then patching the *.m2ts files according to the table.

schluppo

28th October 2008, 03:52

Here's the complete structure of the 'conversion table' (this covers exactly all the data which is given by trap 0x20):

header
16 bit: [number of tables]

then [number of tables] tables, where a table is as follows:

table
32 bit: [table id]
16 bit: [number of segments]

then [number of segments] times:
32 bit: address of each segment
then [number of segments] segments, where a segment is as follows:

segment
32 bit: [number of entries]
then [number of entries] times: 32 bit indexing value
then [number of entries] times: 16 bytes table entry

In the two examples that we could look at so far, the conversion table consists of 0x24 / 0x42 tables, but all other tables apart from the one with [table id]=1 just contain varying numbers of empty segments (i.e. segments with [number of entries]=0).

So here is an example of how to get to "DAT" table 1, segment 3, entry 8. Be aware that all the mentioned addresses are relative to the starting address of the conversion table. Also notice that the counting of tables starts at 1 whereas counting of segments and entries starts at 0:

At 0x00000000, check: 1 <= [number of tables]? If not: Error!
Seek to table_offset for [table_id]=1. In this case, this offset is 0x00000006. For [table id]=2, the offset is 0x000FC930 etc.
At 0x00000006, check: 3 < [number of segments]? If not: Error!
Seek to segment_address for segment 3. In this case, this address lies at 0x00000006 + 2 + 3*4 = 0x00000014 and has value 0x00001FC8.
Seek to 0x00001FC8.
At 0x00001FC8, check: 8 < [number of entries]? If not: Error!
Seek to 0x00001FC8 + 4 + 8*4 = 0x00001FEC.
Get indexing value from 0x00001FEC (in this case, the indexing value is 0x001B0E1E).
Seek to 0x00001FC8 + 4 + [number of entries]*4 + 8*16 = 0x00001FC8 + 4 + 0xBA*4 + 8*16 = 0x00002334.
Get table entry from 0x00002334 (in this case, the table entry is 0xC04C9D38 0xD81F55CD 0x95D64326 0x196C2248).

Edit: Also, [number of segments] of table 1 will tell us, how often we have to invoke event 0x220 to get all the segment keys (DAT: 0x153 times, I Robot: 0x141 times).

Accident

28th October 2008, 04:59

Having a go at the *Slot functions, and I am curious about this comment

The return value is the attached slot number, or an error
code if the operation failed (e.g., because of a code hash mismatch)

However, the JAVA code that is currently checked in only returns STATUS_OK. So which way is more correct?

Edit: Ah ok thanks Oopho2ei, I will follow the code.
Is it not peculiar the offset from conversion table are only 32bit, can it not then scramble beyond 4GB boundary?

schluppo

28th October 2008, 06:58

The debugger is now automatically XORing the entries from the correct segment of the conversion table with the current segment key whenever an event 0x220 is finished. Also it will now take the total number of segments from the conversion table and then invoke event 0x220 as often as necessary, until the complete table is decrypted. Once you press 'run' now, it will run until the conversion table is completely decrypted. The table is only dumped once, when the last event 0x220 was finished. The whole process takes around 30 seconds on my machine.

All that's needed now, is a program which replaces the correct parts of the stream-files with the entries from the conversion table :)

Edit: The conversion table that is generated by the newest version of the debugger coincides with the values from the comparison file in posting #282 (http://forum.doom9.org/showthread.php?p=1206994#post1206994) :)

Edit 2: I figured out three of the four remaining unknown bytes of an entry in the conversion table. Also I figured out the indexing.

The first indexing value in the table is:
0x9900A.
The first entry is:
4A 1A A0 53 64 B1 AA 2D 40 8E 4A 4D EF 4B 6B E5

(0x9900A+0x1AA)*0xC0 + 0x64 = 0x72D4764 // offset in the big file where the first 5 bytes are put to
(0x9900A+0x1AA+0x053)*0xC0 + 0xB1 = 0x72D85F1 // offset in the big file where the second 5 bytes are put to

-----

The second indexing value in the table is:
0x99C85.
The second entry is:
8F 20 C0 11 AA 98 B6 F7 76 EC CB 7E 61 A3 C6 C1

(0x99C85+0x20C)*0xC0 + 0xAA = 0x736ED6A // offset in the big file where the third 5 bytes are put to
(0x99C85+0x20C+0x011)*0xC0 + 0x98 = 0x736FA18 // offset in the big file where the fourth 5 bytes are put to

So, the whole big file is cut down in blocks of 0xC0 bytes. The indexing value points to a section of around 0x300 of these small blocks. Bits 9-20 of the entry select one of the first ~0x26D of these blocks. Bits 21-32 select one of the remaining blocks. The sum of both numbers usually is around 0x1FF except for some cases where the first number is bigger than 0x200. Finally bytes 5 and 6 give the exact position inside the selected blocks.

I don't know the meaning of the first 8 bits of the entries yet (checksum over the other 15 bytes?).

Anyway, it is now possible to code a program which takes a conversion table as input and then goes through a list of .m2ts files, repairing them.

Oopho2ei

28th October 2008, 13:47

Here's the complete structure of the 'conversion table' (this covers exactly all the data which is given by trap 0x20):
Thanks a lot! Indeed there are 36 (=0x24) m2ts files in the BDVM/STREAM directory. Starting with 00001.m2ts (the main movie) up to 00036.m2ts. So it seems there is a table for every m2ts file. The first file which is played is 00005.m2ts and then the movie starts (00001.m2ts). Is the table id in the table of the first TRAP_Schluppo 5 or is it exactly the same table?

Also thanks for your work on the debugger.

Having a go at the *Slot functions, and I am curious about this comment

The return value is the attached slot number, or an error
code if the operation failed (e.g., because of a code hash mismatch)

However, the JAVA code that is currently checked in only returns STATUS_OK. So which way is more correct?
Someone has to update the documentation. It's mostly a copied from the patent (which describes a similar system). If in doubt trust the source code. :rolleyes:

schluppo

28th October 2008, 14:07

Is the table id in the table of the first TRAP_Schluppo 5 or is it exactly the same table?

The 'sub-tables' in the conversion table should be lexically ordered, that means, the table for 00001.m2ts should be the first table (with smallest ID) and the table for 00036.m2ts should be the last table (with biggest ID). I assume that the first call of trap 0x20 gives exactly the same conversion table as the second call (but I did not verify that yet).

Also, please take note of my editions of posting #287 :)

Edit: I just thought, that we may need to invoke event 0x220 (0,m,n) with <m>!=1. My guess is, that <m> selects the 'sub-table' number and <n> selects the segment number. The result is then the segment key for the selected segment in the selected 'sub-table'. For <m>!=1, this should allow us to decrypt the 'sub-tables' for other files than 00001.m2ts (could be necessary for e.g. "Die Another Day"). However, for DAT and I Robot, this should be no problem yet, since these movies only use one encrypted file (00001.m2ts).

Oopho2ei

28th October 2008, 14:19

Also, please take note of my editions of posting #287 :)
Well done! :thanks:
I will try to repair 00001.m2ts of "The Day After Tomorrow" now. :)

frogman

28th October 2008, 16:58

The light of day now appear to be coming up on those sony+fox cockroaches. hmm... I think I am having a good day.

Ps: Slysoft just released an open beta "AnyDvD HD"

Feedback is welcome from everybody else.

http://forum.slysoft.com/showthread.php?t=21699
http://sandbox.slysoft.com/beta/SetupAnyDVD6478.exe

6.4.7.8 2008 10 22
- "New (Blu-ray): Added option to disable BD-Live"
- New (Blu-ray): Added removal of region locks from menus
- New (Blu-ray): Added support for new version of the BD+ copy protection
- Some minor fixes and improvements
- Updated languages

Oopho2ei

28th October 2008, 19:28

I am glad to announce the first successful restoration of the BD+ protected movie "The Day After Tomorrow" in linux. It was done using a blue ray drive with patched firmware (to get the volume id), DumpHD to decrypt the contents according to the AACS specification and the BDVM debugger from this thread to generate the conversion table. The conversion table is the key information to successfully repair all the broken parts in m2ts files to restore the original video content. This small tool (http://uploaded.to/?id=9ale4l) was finally used to repair the main movie file "00001.m2ts" according to the conversion table.

To verify the correctness i compared my 00001.m2ts with the one AnyDVD-HD creates and they both match. The MD5 hash of this 30GB large file is in both cases "0fa2bc65c25d7087a198a61c693a0a72".

Well done everyone! :)

RunningSkittle

28th October 2008, 22:31

Looks like its almost time for me to go get a BR drive :)

LoRd_MuldeR

28th October 2008, 23:05

Congratulations :D

I don't have a BR drive, but it's always nice to know that there's a cure for the DRM disease.

chavonbravo

29th October 2008, 00:05

With this working, we're pretty much at the level of anydvd hd, as long as it's mkbv7 or below, right?

Oopho2ei

29th October 2008, 00:06

Thanks. I hope that other people try to write their own implementations and make their source code public. The debugger source code and the information from this thread and the repository should be enough to provide basic BD+ support. I am no professional programmer and it might also be better to integrate the core functionality of the debugger into DumpHD instead of having lots of smaller tools.

- New (Blu-ray): Added support for new version of the BD+ copy protection
I've updated to the latest firmware version (sept. 2008) and couldn't find any major changes in my player. No new command or traps. Not even new AES keys or a different ECDSA certificate.
I believe this statement means the BD+ implementation in AnyDVD-HD is buggy/incomplete (like ours) and new versions of the content code are handled incorrectly because for example a particular trap was never used before (or never used with certain parameters) and was therefor insufficiently tested or not implemented (like TRAP_RunNative).

With this working, we're pretty much at the level of anydvd hd, as long as it's mkbv7 or below, right?
I currently can't answer that question because i only have 2 BD+ protected movies and only one of them (The Day After Tomorrow) has been tested so far. There is still some work to be done. Also using the debugger to get the conversion table and then repair every m2ts file with the small c program i wrote is too complicated for the average user. I am hoping other people will like KenD00 will integrate the code into their tools to make watching BD+ protected movies in linux possible for the average user.

Is it not peculiar the offset from conversion table are only 32bit, can it not then scramble beyond 4GB boundary?
Sorry i didn't see you've edited your posting. You will find the answer when you look at the way the address is calculated line 151-156 (http://uploaded.to/?id=9ale4l). As you can see it's adjusted with a positive value and then multiplied with 0xC0 (the buffer size). I haven't calculated the highest possible address but it's obviously much larger than 2^32. Also take a look at Schluppos example in posting #287 (http://forum.doom9.org/showpost.php?p=1207409&postcount=287).

kkloster21

29th October 2008, 00:13

awesome news! looking forward to seeing you guys release a tool for this! (whether its a separate package or it gets integrated into dumpHD)

a big thanks to Oopho2ei, bmnot, schluppo, Disabled, evdberg and anybody else who contributed!

I regret that i don't have the knowledge to be of any help in the programming but i do have several BD+ discs and a blu-ray drive that i more than happy to test on! let me know when to start!

tteich

29th October 2008, 07:55

I am glad to announce the first successful restoration of the BD+ protected movie "The Day After Tomorrow" in linux. It was done using a blue ray drive with patched firmware (to get the volume id), DumpHD to decrypt the contents according to the AACS specification and the BDVM debugger from this thread to generate the conversion table. The conversion table is the key information to successfully repair all the broken parts in m2ts files to restore the original video content. This small tool (http://uploaded.to/?id=9ale4l) was finally used to repair the main movie file "00001.m2ts" according to the conversion table.

To verify the correctness i compared my 00001.m2ts with the one AnyDVD-HD creates and they both match. The MD5 hash of this 30GB large file is in both cases "0fa2bc65c25d7087a198a61c693a0a72".

Well done everyone! :)
Congratulations!

Wimpy

29th October 2008, 09:20

I am hoping other people will like KenD00 will integrate the code into their tools to make watching BD+ protected movies in linux possible for the average user.

Excellent work by all involved!

I don't "do" Windows. I have been tinkering with various tools, on and off and with varying success, for over a year now so that I can watch my Blu-Ray films on my Linux computers.

Looking forward the the BD+ tools to getting integrated in DumpHD. I'm fortunate enough to own a GGW-H20L which I flashed with your patched firmware last night. Looking forward to testing the result, on non-BD+ disks, this evening when I am back from work.

Many thanks to all involved, you are helping to get the Linux community on an even footing with regard to Blu-Ray support!

Wombler

29th October 2008, 09:37

That's fantastic news guys and congratulations to everyone involved.

It's a very significant achievement. :cool::cool::cool:

Wombler

zeroprobe

29th October 2008, 11:23

There are lots of talented people in here. Kudos to everyone involved. I feel stupid reading some of your posts lol.

Oopho2ei

29th October 2008, 17:43

Like Schluppo pointed out the event replay we currently use to get the decrypted conversion table won't work if the main movie is distributed over multiple m2ts files. So we should try now to identify the remaining unknown events by taking a closer look at the "answers" from the content code. I have made a small list (http://svn.assembla.com/svn/bdplus/trunk/doc/events/overview.txt) of what we currently have identified.

For those who don't know what events are: They are major changes of which the content code has to be informed. For example the user pressed the 'play' button or the 'stop' button.

Furthermore i made a description (http://svn.assembla.com/svn/bdplus/trunk/doc/conversion_table/structure.txt) of the conversion table syntax (including the non formal inital description from Schluppo).

We also still need a "real" name for TRAP_#0020 (currently called TRAP_Schluppo :D). This trap seems to send the conversion table to the player (UINT32 length, UINT32 offset). The correct name has to fit alphabetically between TRAP_Finished and TRAP_MediaReadFile: 0010 TRAP_Finished
0020 TRAP_... ?
0540 TRAP_MediaReadFile
I would suggest a slightly modified version of the name Schluppo proposed:
TRAP_GetConversionTable( UINT32 length, UINT32 offset );
Let me know if you agree.

schluppo

29th October 2008, 23:11

Some news on the events:

First: When event 0x220(0,2,n) happens (in "DAT"), the VM is just doing the trap testing but does not try to attach slot 0 and then read from it. Also the returned segment key is 16 bytes of 0x00 in this case. The same holds for event 0x220(0,0,n).

Also, the first four bytes after the event-id seem to be 0x00000000 for every event, so I think, these bytes are actually not a parameter to any event.

The following is a list of observations on events (the list also includes a lot of interpretation):

event 0x220 = EVENT_GetSegmentKey(UINT32 FileID, UINT32 SegmentNumber)

This event first tests parts of the trap-system of the VM (the ids/parameters of the testing trap-calls depend on
FileID and SegmentNumber in no recognizable way). After the tests were successfully passed, and if FileID and
SegmentNumber specify a valid, encrypted segment of the conversion table, the event will attach slot 0 and read from it.
Using the data which was read from slot 0, the SegmentKey for the specified segment is computed (without the use of
further traps) and stored at VM-memory 0x10. In the case, that FileID and SegmentNumber select an invalid file (i.e.
the FileID does not occur in the conversion table or it occurs in the conversion table but has no non-empty segment)
or an invalid segment (i.e. the SegmentNumber is larger than the number of segments for the according FileID), the event
writes 16 times the byte 0x00 to VM-memory at address 0x10.

----

event 0x010 = EVENT_Shutdown(void)

This event is probably signaled by the player whenever the eject-button is pressed and the player is getting ready to finish
playback of the movie.

This event terminates the execution of the VM, most probably including the destruction of the conversion table.

----

event 0x110 = EVENT_BuildConversionTable(UINT32 TableID?)

This event is probably signaled by the player whenever the play-button is pressed and the player is getting ready to start
playback of the movie.

If TableID = 0xFFFF or 0, the conversion table is reset (to length 0 at VM-memory 0x1000). My guess is, that enough memory to
hold the conversion table is reserved and filled with 0xFF in this case (effectively overwritting the table, in the
case that it previously resided in VM-memory).

If TableID has a different value (including the cases that TableID = 1, 2 etc.), the VM extensively tests itself (by
test-calling most of the traps, each several hundred times). After the testing, the complete (but still encrypted) conversion
table is read from one of the .svm-files (usually 00001.svm) and rebuilt (without the use of further traps). The rebuilt
conversion table still has encrypted repair descriptors.

After the VM is done with handling this event, the new location and length of the conversion table is signaled to the player
by using TRAP_GetConversionTable.

----

event 0x210 = EVENT_CheckDrive(UINT32 OpID?)

I don't have much information about this event. When OpID = 1, the VM will execute TRAP_DeviceDiscovery(2,1) and then
TRAP_DeviceDiscovery(0,1). If OpID != 1 (especially the cases that OpID = 0 or 2), the VM will directly stop
execution with TRAP_Finished.

Furthermore, the order of events is important. It is for instance not possible to get segment keys before the conversion table has been built. On the other hand, I noticed, that it is not necessary to execute EVENT_BuildConversionTable(1) twice before executing EVENT_GetSegmentKey(1,n).

So here is the reasonable order of events:

- Let the VM initialize itself until it reaches TRAP_Finished for the first time.
- EVENT_BuildConversionTable(0xFFFF);
- EVENT_CheckDrive(1);
- EVENT_BuildConversionTable(1);
- EVENT_GetSegmentKey(1,1);
...
- EVENT_GetSegmentKey(1,n_1);
- EVENT_GetSegmentKey(2,1);
...
- EVENT_GetSegmentKey(2,n_2);
...
...
- EVENT_GetSegmentKey(m,1);
...
- EVENT_GetSegmentKey(m,n_m);
(- EVENT_Shutdown() )

<m> is the number of encrypted files, <n_i> is the number of segments in the file with id <i>.

In general, if the event parameters are ridiculously strange (such as starting event 0x220 with just one parameter given and the other parameter left undefined), the VM will just immediately terminate execution and not allow successful invocation of any more events. If the parameters are just a little off, the VM may show slightly different behaviour but still accept further events.

Oopho2ei

30th October 2008, 00:26

Thanks for the analysis. It contains some interesting details. Regard the names you have (quickly) proposed i suggest names which describe the event and not how the content code of this particular movie reacts to it. For example the content code would likely execute a "GetSegmentKey" function in the event of a segment switch (event #0220 ?) in the movie or execute a "BuildConversionTable" function in the event of starting playback of a m2ts file (event #0110 ?).

The interface defines 9 callbacks (Events), which devices may invoke to notify content code of outside events, or to request specific actions.

Keep in mind that the first file which is played back (00005.m2ts) is a short animation of the FOX logo which is not BD+ obfuscated. After that the playback of the movie starts (00001.m2ts).

Edit: It would be ok if you replace "EVENT" with "CALLBACK" if you think it's a specific callback function called by the player and not a general outside event for which different versions of content code can behave differently. For example event #0220 = CALLBACK_GetSegmentKey is ok. :)

Edit: I also wonder if the event identifiers are grouped like the trap identifiers. Maybe we have a group 0 (0x0000 and 0x0010), group 1 (0x0110) and a group 2 (0x0210 and 0x0220). So you could check if the events/callbacks within one group have something in common.

schluppo

30th October 2008, 08:21

Feel free to heavilly change the proposed names of the EVENTS / CALLBACKS :) I just used them to describe what the current content code seems to be doing upon invocation of the respective EVENT / CALLBACK.

Apart from that, I did some testing and noticed, that for "DAT", the content code is generating the correct conversion table AND correct segment keys, even if there is no VolumeID used (the VolumeID is set to 0x00000000 0x00000000 0x00000000 0x00000000 in this case). Of course I could not test the complete conversion table since I do not have the "00001.m2ts" and hence could not try to repair it using the conversion table which was created WITHOUT VolumeID. Still I came to the conclusion (or 'hope'), that it is possible to remove BD+ from at least the first bunch of released titles without using any VolumeID! Things may have changed starting with "Jumper" or later titles. Can you verify my observation?

Also, I tried to run the debugger on "I Robot", but did not get far, since I do not have the file "00012.m2ts" and the according SHA-Checksum is hence wrong. So the VM just called TRAP_Finished and stopped execution after ~1500 traps (not accepting any events). However, my guess is, that construction of the conversion table and obtainment of the segment keys should work fine, once this SHA-Checksum is correct. Could you try to repair "I Robot", Oopho?

Oopho2ei

30th October 2008, 21:29

This is the latest binary snapshot of the debugger: BDVM Debugger (testing) (http://uploaded.to/?id=ku4wpf) *updated*

These steps are supposed to guide you getting the decrypted conversion table for your BD+ protected disc. You don't need any other tools for this.
Preparations:
-------------

For windows you need the BouncyCastle security provider for java to perform ECDSA authentication to the content code.
Look at these postings for more details on how to install it:
- http://forum.doom9.org/showpost.php?p=1195864&postcount=218
- http://forum.doom9.org/showpost.php?p=1208775&postcount=315

Follow these steps please:
--------------------------

0. mount the disc (e.g.: "mount /dev/sr0 /media/cdrom"). You should see a BDSVM directory otherwise the disc is not
BD+ protected and you should try a different movie.
1. start the debugger with "sh debugger.sh" from this directory
2. select "Settings -> Set disc mount point" and navigate to the mount point of your blue ray disc (e.g.: /media/cdrom/)
3. if you haven't edited the volume_id.bin with a hex editor to match the volume ID from this disc select
"Settings -> set volume id" and enter the 16 byte number as hex (e.g.: "C659019F89718033F0FC54E894C98428")
If you don't know the volume id you can either obtain it from the output of aacskeys or skip this point because
the content code seems to ignore the volume id for the discs we checked.
4. press run or hit the "F5" key and watch the console output
5. after around 30s you should find a file named "conv_tab.bin" containing the decrypted conversion table in this
directory.
6. Post the "conv_tab.bin" and the console output in the forum please.
You will find a small readme in the archive which contains the same description. Extract the archive in a directory, navigate to this directory and launch the debugger from there.

If you would like to test the resulting conversion table you need to decrypt the disc with DumpHD and then use this tool (http://uploaded.to/?id=9ale4l) to repair the 00001.m2ts.

Thanks for your reports in advance.

@Schluppo: i get a conversion table for i robot and the decryption of the same seems to work fine. I currently have some problems with anydvd which is why i couldn't verify this yet. This issue will probably be solved soon. It doesn't seem to know my drive has it's firmware patched.

Oopho2ei

31st October 2008, 02:31

I have now successfully repaired "I robot". My 00001.m2ts and the one created by AnyDVD-HD are identical. There was however one bug which caused segment 0 of the first subtable in the conversion table never to be decrypted. The bug didn't show up with "The Day After Tomorrow" because that particular segment was empty there. :rolleyes:
Anyway the bug is fixed in revision 116 and i have also updated the binary snapshot archive in the above posting. Furthermore for those who are curious of how these conversion tables look like: these (http://uploaded.to/?id=snt7ho) are the conversion tables i used to successfully repair both movies.

Edit: i believe the checking of 00001.m2ts with TRAP_MediaSHAFileHash is because a particular version of a software player called "PowerDVD" allowed the playback a BD+ protected (but AACS decrypted) movies from hard drive. So those checks probably try to make sure the movie is still encrypted and therefor cannot be played from hard drive with this player (without hacks of course).

Edit: feel free to play around with the content code. If you get any other conversion tables for both movies than those i posted above something went wrong. :p

Turtleggjp

31st October 2008, 18:29

Very nice work guys! So now that you have figured out most of BD+'s mysteries, what can they do to protect future releases? With AACS, it's easy, just revoke the keys used. Does your implementation of BD+ use any revokable keys, or are they limited to the unused instructions to try and fool you in the future?

schluppo

31st October 2008, 18:50

- They could find bugs in our basic VM (such as treatment of divide by zero or similar) and then exploit the bugs in order to distinguish the emulated VM from a valid player. Anyway, our basic VM is behaving fine for hundreds of millions of instruction executions already, so they might have a hard time, to find any bugs.
- They can use previously unused traps or they might use previously used traps with unusual parameters. However, our trap-implementations are already close to the standard-implementation of traps, and it is easy to fix any bugs in our traps. So over time (as the VM is getting closer and closer to the standards), the BD+ developers will have trouble distinguishing our VM from a valid player by just looking at the correct treatment of traps.
- The current implementation of traps is using player specific data such as player keys (for AES), the player's public key (for ECDSA), memory footprints and player signatures. The BD+ developers can force a firmware update (including new player keys, public key, memory footprints or signatures) and then use the content code to 'revoke' (or rather interdict) the leaked player specific data in future. As I see it, this is the only tactic which may cause trouble in the long run, since it might become necessary to obtain the player specific data for every new version of BD+. I guess the situation could become similar to AACS.

Oopho2ei

31st October 2008, 18:55

Very nice work guys! So now that you have figured out most of BD+'s mysteries, what can they do to protect future releases? With AACS, it's easy, just revoke the keys used. Does your implementation of BD+ use any revokable keys, or are they limited to the unused instructions to try and fool you in the future?
The key set currently in use consists of 7 AES keys for TRAP_Aes (see posting #163 (http://forum.doom9.org/showpost.php?p=1187564&postcount=163)) and two ECDSA public/private key pairs for TRAP_PrivateKey (see posting #188 (http://forum.doom9.org/showpost.php?p=1190187&postcount=188)). These keys can theoretically be revoked (e.g. by adding 7 more AES keys and don't use the old ones anymore for new movies). Apart from that the purpose of the program (called "content code") running inside the player on a virtual machine is to detect any known compromised players or known unlicensed emulators (like ours). The content code is give a wide range of opportunities to do that. For example it has (limited) access to the player memory and can even execute arbitrary code on the machine though we haven't seen that yet and our emulator doesn't support this either.
As long as we have access to a working (licensed) players all these measures are useless as we can record traces from this player and adjust the data "injected" in the virtual machine address space by traps or events to perfectly match our recordings. Even if whitebox attack resistant AES or ECDSA algorithms are used and nobody manages to break them we can still use the obfuscated algorithms and their keys.

Edit: this project has left it's marks in wikipedia (http://en.wikipedia.org/wiki/BD%2B)

KenD00

1st November 2008, 00:24

First i have to say you are doing amazing work here, thank you. I have to admit that i don't understand much of what you are doing here, but the conversion table, thats something i can handle :D.

So i wanted to test this one out. Unfortunately i don't have any of the movies you are using here but two different ones, Die Hard 4 and Jumper. I have used the latest debugger binary package (maybe you should add to the readme that you need the BouncyCastle provider to run it) together with the correct volume id files. But somehow i'm not successful :(.

Jumper failed quite fast, it produced the error "Unknown trap". For Die Hard 4 i got a conversion table but along with a lot of error messages about missing subtable 0001. I have uploaded that conversion table and the console output here (http://www.sendspace.com/file/97q9p7).

Because im not so good at reading binary files i wrote a small viewer which decodes conversion tables and displays them more human readable, ConvTableView 0.1 (http://www.sendspace.com/file/cprbca). My conversion table doesn't look so bad after all, an entry for every m2ts file, all entries are empty except for the main movie which has about 400 segments.

I modified the content repair program to fix that movie (it uses tableid 20) and changed it to copy the conversion table into memory to reduce seeking too speed it up (but it doesn't seem to be much faster now), you can download it here (http://www.sendspace.com/file/7y7kgx).

However, after applying the fix the movie is still broken and somehow it looks worse than before the patching :(.

:rolleyes:

Oopho2ei

1st November 2008, 01:13

maybe you should add to the readme that you need the BouncyCastle provider to run it
I will add this to the readme. Do you know if i could somehow include the BouncyCastle library in our jar?

Jumper failed quite fast, it produced the error "Unknown trap".
Whoever wrote that error message could have at least included the trap id. :rolleyes: I will suppress the error message and instead simply return 0x80FFFFFF like my player (maybe with a warning).

For Die Hard 4 i got a conversion table but along with a lot of error messages about missing subtable 0001.
That's because the current version of the decoder module of the debugger always tries to get all the segment keys for the first subtable (0001) assuming it is the movie. It is encouraging however that you get the conversion table though it is never being decrypted. I will try to load your encrypted conversion table and see if i can make the debugger navigate correctly over all the subtables.

Because im not so good at reading binary files i wrote a small viewer which decodes conversion tables and displays them more human readable, ConvTableView 0.1 (http://www.sendspace.com/file/cprbca).
Looks great. Thank you. :)

However, after applying the fix the movie is still broken and somehow it looks worse than before the patching :(.
Yeah because the debugger never tried to decrypt subtable 20. Instead it tried to decrypt subtable 1 which doesn't even exist. :rolleyes:

I will try to fix it now.

Edit: i have tried to fix it but the source code is a mess so i have decided to rewrite all the conversion table related code (again :rolleyes:). I believe it is important for this reference source code to be plain and simple so i will introduce extra classes for segments and subtables. This should be finished soon.... i hope :)

kkloster21

1st November 2008, 04:41

Hey guys, I tried the debugger on a couple of BD+ discs but it seems clear that i am missing something. never got any conv_tab.bin file. here is what i kept getting from the console:

$ java -jar Debugger.jar
[W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /media/cdrom1/BDSVM/00000.svm ...
Volume ID set to: 98 62 39 55 14 61 94 83 D2 8A B9 B5 C3 B4 BA E8
java.security.NoSuchProviderException: no such provider: BC
at sun.security.jca.GetInstance.getService(GetInstance.java:83)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:206)
at java.security.Signature.getInstance(Signature.java:341)
at bdvm.bdsvm_player_traps.PrivateKey(bdsvm_player_traps.java:486)
at bdvm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.java:632)
at bdvm.BDVM.step(BDVM.java:335)
at bdvm.Debugger.step(Debugger.java:753)
at bdvm.Debugger.run(Debugger.java:881)
at bdvm.Debugger.actionPerformed(Debugger.java:344)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2012)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2335)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:404)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:253)
at java.awt.Component.processMouseEvent(Component.java:6106)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3276)
at java.awt.Component.processEvent(Component.java:5871)
at java.awt.Container.processEvent(Container.java:2105)
at java.awt.Component.dispatchEventImpl(Component.java:4467)
at java.awt.Container.dispatchEventImpl(Container.java:2163)
at java.awt.Component.dispatchEvent(Component.java:4293)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4461)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4125)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4055)
at java.awt.Container.dispatchEventImpl(Container.java:2149)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4293)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:604)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:275)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:200)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:190)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:185)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:138)
Exception in thread "AWT-EventQueue-0" java.lang.Error: Unkown Trap
at bdvm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.java:868)
at bdvm.BDVM.step(BDVM.java:335)
at bdvm.Debugger.step(Debugger.java:753)
at bdvm.Debugger.run(Debugger.java:881)
at bdvm.Debugger.actionPerformed(Debugger.java:344)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2012)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2335)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:404)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:253)
at java.awt.Component.processMouseEvent(Component.java:6106)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3276)
at java.awt.Component.processEvent(Component.java:5871)
at java.awt.Container.processEvent(Container.java:2105)
at java.awt.Component.dispatchEventImpl(Component.java:4467)
at java.awt.Container.dispatchEventImpl(Container.java:2163)
at java.awt.Component.dispatchEvent(Component.java:4293)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4461)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4125)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4055)
at java.awt.Container.dispatchEventImpl(Container.java:2149)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4293)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:604)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:275)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:200)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:190)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:185)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:177)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:138)

and it just sits there. i'd like to give you guys more useful feedback/results than this but it looks i am lacking something. what can i do?

thanks to the guys in thread who are working so hard.

schluppo

1st November 2008, 08:52

You need the BouncyCastle provider installed, your error occurs since you don't have it installed. See post here (http://forum.doom9.org/showthread.php?p=1195864#post1195864) for more details about how to install it.

Thanks for your help :)

frogman

1st November 2008, 14:38

You need the BouncyCastle provider installed, your error occurs since you don't have it installed. See post here (http://forum.doom9.org/showthread.php?p=1195864#post1195864) for more details about how to install it.

Thanks for your help :)

Without the developer environment installed you can use
C:\Program Files\Java\jre1.6.0_07\lib\security folder

Should still work. Vista 32

kkloster21

1st November 2008, 14:52

i installed the bouncy castle provider and this was my new console output:

$ java -jar Debugger.jar
[W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /media/cdrom1/BDSVM/00000.svm ...
Volume ID set to: 98 62 39 55 14 61 94 83 D2 8A B9 B5 C3 B4 BA E8
[Event #00000001] 0110 ( 00000000, 0000FFFF )
[Event #00000002] 0210 ( 00000000, 00000001 )
[Event #00000003] 0110 ( 00000000, 00000001 )
Conversion table set
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000004] 0220 ( 00000000, 00000001, 00000000)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000005] 0220 ( 00000000, 00000001, 00000001)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000006] 0220 ( 00000000, 00000001, 00000002)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000007] 0220 ( 00000000, 00000001, 00000003)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000008] 0220 ( 00000000, 00000001, 00000004)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #00000009] 0220 ( 00000000, 00000001, 00000005)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #0000000A] 0220 ( 00000000, 00000001, 00000006)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #0000000B] 0220 ( 00000000, 00000001, 00000007)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
.
.
.
[Event #000001B5] 0220 ( 00000000, 00000001, 000001B1)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #000001B6] 0220 ( 00000000, 00000001, 000001B2)
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[E] getSubtableOffset: couldn't find subtable 0001
[Event #000001B6] 0010 ( 00000000, 00000001 )

i had to remove a bunch of the output but it looked similar. hopefully this is enough.

but it did produce a conv_tab.bin that is about 961.4 KB.

@frogman: i am running linux.

Oopho2ei

1st November 2008, 15:52

I have rewritten all the conversion table related code and parts of the decoder. It basically scans the conversion table now for nonempty segments which are marked as encrypted. If it finds such a segment it will ask the content code for the segment key and decrypt it clearing the encrypted flag for the segment. The code supports decrypting the segments in a random order although it currently decrypts them in the order they appear in the conversion table. I have loaded the conversion table from KenD00 and it seems it is handled correctly now. Now let's hope the second parameter of event #0220 really is the subtable id. :)

This is a new binary snapshot for testing: Debugger (testing) (http://uploaded.to/?id=q4e22r)

Edit: event #0110 still always passes as second parameter '0x00000001' which worked for "The Day After Tomorrow" and "I Robot". We assume this parameter is the table id. All this doesn't matter as long as you get the conversion table (encrypted) which you can tell from the console message "Conversion table set". Sadly i don't have any BD+ protected disc where any file other than 00001.m2ts contains the movie.

Edit: i've added the BouncyCastle security provider library to the package. You should start the debugger with "sh debugger.sh" now.

Edit: I have removed the BouncyCastle library again as it still seems to require manually editing the java.security.

kkloster21

1st November 2008, 16:28

i think it worked with your latest snapshot Oopho! I got the "Conversion table set" message:

$ java -jar Debugger.jar [W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /media/cdrom1/BDSVM/00000.svm ...
Volume ID set to: 98 62 39 55 14 61 94 83 D2 8A B9 B5 C3 B4 BA E8
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[Event #00000002] 0110 ( 00000000, 00000001 )
Conversion table set
[Event #00000003] 0220 ( 00000000, 00000000, 00000000)
[Event #00000004] 0220 ( 00000000, 00000000, 00000001)
[Event #00000005] 0220 ( 00000000, 00000000, 00000002)
[Event #00000006] 0220 ( 00000000, 00000000, 00000003)
[Event #00000007] 0220 ( 00000000, 00000000, 00000004)
[Event #00000008] 0220 ( 00000000, 00000000, 00000005)
[Event #00000009] 0220 ( 00000000, 00000000, 00000006)
[Event #0000000A] 0220 ( 00000000, 00000000, 00000007)
[Event #0000000B] 0220 ( 00000000, 00000000, 00000008)
[Event #0000000C] 0220 ( 00000000, 00000000, 00000009)
[Event #0000000D] 0220 ( 00000000, 00000000, 0000000A)
[Event #0000000E] 0220 ( 00000000, 00000000, 0000000B)
[Event #0000000F] 0220 ( 00000000, 00000000, 0000000C)
[Event #00000010] 0220 ( 00000000, 00000000, 0000000D)
[Event #00000011] 0220 ( 00000000, 00000000, 0000000E)
[Event #00000012] 0220 ( 00000000, 00000000, 0000000F)
[Event #00000013] 0220 ( 00000000, 00000000, 00000010)
[Event #00000014] 0220 ( 00000000, 00000000, 00000011)
[Event #00000015] 0220 ( 00000000, 00000000, 00000012)
[Event #00000016] 0220 ( 00000000, 00000000, 00000013)
[Event #00000017] 0220 ( 00000000, 00000000, 00000014)
[Event #00000018] 0220 ( 00000000, 00000000, 00000015)
[Event #00000019] 0220 ( 00000000, 00000000, 00000016)
[Event #0000001A] 0220 ( 00000000, 00000000, 00000017)
[Event #0000001B] 0220 ( 00000000, 00000000, 00000018)
[Event #0000001C] 0220 ( 00000000, 00000000, 00000019)
[Event #0000001D] 0220 ( 00000000, 00000000, 0000001A)
[Event #0000001E] 0220 ( 00000000, 00000000, 0000001B)
[Event #0000001F] 0220 ( 00000000, 00000000, 0000001C)
[Event #00000020] 0220 ( 00000000, 00000000, 0000001D)
[Event #00000021] 0220 ( 00000000, 00000000, 0000001E)
[Event #00000022] 0220 ( 00000000, 00000000, 0000001F)
[Event #00000023] 0220 ( 00000000, 00000000, 00000020)
[Event #00000024] 0220 ( 00000000, 00000000, 00000021)
[Event #00000025] 0220 ( 00000000, 00000000, 00000022)
[Event #00000026] 0220 ( 00000000, 00000000, 00000023)
[Event #00000027] 0220 ( 00000000, 00000000, 00000024)
[Event #00000028] 0220 ( 00000000, 00000000, 00000025)
[Event #00000029] 0220 ( 00000000, 00000000, 00000026)
[Event #0000002A] 0220 ( 00000000, 00000000, 00000027)
[Event #0000002B] 0220 ( 00000000, 00000000, 00000028)
[Event #0000002C] 0220 ( 00000000, 00000000, 00000029)
[Event #0000002D] 0220 ( 00000000, 00000000, 0000002A)
[Event #0000002E] 0220 ( 00000000, 00000000, 0000002B)
[Event #0000002F] 0220 ( 00000000, 00000000, 0000002C)
[Event #00000030] 0220 ( 00000000, 00000000, 0000002D)
[Event #00000031] 0220 ( 00000000, 00000000, 0000002E)
[Event #00000032] 0220 ( 00000000, 00000000, 0000002F)
.
.
.
[Event #000001AA] 0220 ( 00000000, 00000000, 000001A7)
[Event #000001AB] 0220 ( 00000000, 00000000, 000001A8)
[Event #000001AC] 0220 ( 00000000, 00000000, 000001A9)
[Event #000001AD] 0220 ( 00000000, 00000000, 000001AA)
[Event #000001AE] 0220 ( 00000000, 00000000, 000001AB)
[Event #000001AF] 0220 ( 00000000, 00000000, 000001AC)
[Event #000001B0] 0220 ( 00000000, 00000000, 000001AD)
[Event #000001B1] 0220 ( 00000000, 00000000, 000001AE)
[Event #000001B2] 0220 ( 00000000, 00000000, 000001AF)
[Event #000001B3] 0220 ( 00000000, 00000000, 000001B0)
[Event #000001B4] 0220 ( 00000000, 00000000, 000001B1)
[Event #000001B5] 0220 ( 00000000, 00000000, 000001B2)
[Event #000001B6] 0010 ( 00000000, 00000000 )

and it produced a conv_tab.bin file.

note to Oopho2ei that the main movie file was 00000.m2ts (not 00001.m2ts like yours).

now i just need to figure out how to use the conv_tab.bin to watch the BD+ discs. I saw that you posted a tool for this above, is there a readme for using it? also, is it possible to repair it on the fly (rather than ripping the entire disc) and pipe it to mplayer (like we do already with dumpHD) ?

thanks for your terrific and rapid work!

Oopho2ei

1st November 2008, 17:27

i think it worked with your latest snapshot Oopho! I got the "Conversion table set" message:
Looks good but the resulting conversion table can still be wrong if the content code didn't return the correct segment keys (e.g. because i maybe didn't set all the callback/event parameters correctly or in other words "asked the wrong questions"). My player performs some checks on the header of every repair descriptor (e.g. checking if the two most significant bits of the first byte are either '01' or '10'.) which i haven't documented yet. Maybe it's possible to tell if a repair descriptor is correct or not by looking at the header.

now i just need to figure out how to use the conv_tab.bin to watch the BD+ discs. I saw that you posted a tool for this above, is there a readme for using it? also, is it possible to repair it on the fly (rather than ripping the entire disc) and pipe it to mplayer (like we do already with dumpHD) ?
I hope KenD00 will report his results soon. If he is successful he will probably modify his DumpHD to use the decrypted conversion table and repair the movie while decrypting it.
The sample code i posted is only to repair the 00001.m2ts movie file with the subtable 0001. You would have to modify the source (which should be easy because almost every line is commented). You can compile it simply with "gcc -o test content_repair.c". Instead of "test" you can use any other name like "bdplusrepair", "bdplusminus" or whatever.

thanks for your terrific and rapid work!
Thank you for your report! Do you have any other BD+ protected discs you can test?

Edit: you can also try to open the conv_tab.bin with KenD00's ConvTableView 0.1 (http://www.sendspace.com/file/cprbca).

kkloster21

1st November 2008, 21:25

i successfully got a table from another BD+ disc!

$ java -jar Debugger.jar
[W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /media/cdrom1/BDSVM/00000.svm ...
Volume ID set to: 8F 64 CC B3 A1 93 B6 F0 43 0C A1 FD 94 DF 74 9A
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[Event #00000002] 0110 ( 00000000, 00000001 )
Conversion table set
[Event #00000003] 0220 ( 00000000, 00000074, 00000001)
[Event #00000004] 0220 ( 00000000, 00000074, 00000002)
[Event #00000005] 0220 ( 00000000, 00000074, 00000003)
[Event #00000006] 0220 ( 00000000, 00000074, 00000004)
[Event #00000007] 0220 ( 00000000, 00000074, 00000005)
[Event #00000008] 0220 ( 00000000, 00000074, 00000006)
[Event #00000009] 0220 ( 00000000, 00000074, 00000007)
[Event #0000000A] 0220 ( 00000000, 00000074, 00000008)
[Event #0000000B] 0220 ( 00000000, 00000074, 00000009)
[Event #0000000C] 0220 ( 00000000, 00000074, 0000000A)
[Event #0000000D] 0220 ( 00000000, 00000074, 0000000B)
[Event #0000000E] 0220 ( 00000000, 00000074, 0000000C)
[Event #0000000F] 0220 ( 00000000, 00000075, 00000001)
[Event #00000010] 0220 ( 00000000, 00000075, 00000002)
[Event #00000011] 0220 ( 00000000, 00000075, 00000003)
[Event #00000012] 0220 ( 00000000, 00000075, 00000004)
.
.
.
[Event #000000E0] 0220 ( 00000000, 00000077, 0000004B)
[Event #000000E1] 0220 ( 00000000, 00000077, 0000004C)
[Event #000000E2] 0220 ( 00000000, 00000077, 0000004D)
[Event #000000E3] 0220 ( 00000000, 00000077, 0000004E)
[Event #000000E4] 0010 ( 00000000, 00000077 )

also, this disc has the movie split across several .m2ts files that are numbered something like 00116.m2ts, 00117.m2ts, 00118.m2ts, etc. it seems to be working!

also, good to hear from you arnezami! i know everyone here appreciates all your pioneering work on aacskeys.

kkloster21

1st November 2008, 21:34

and another BD+ disc with a conv_tab.bin table successfully created:

$ java -jar Debugger.jar
[W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /media/cdrom1/BDSVM/00000.svm ...
Volume ID set to: AB 82 C6 51 E6 0C 66 91 D4 BC 70 46 D0 5C 70 09
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[Event #00000002] 0110 ( 00000000, 00000001 )
Conversion table set
[Event #00000003] 0220 ( 00000000, 00000001, 00000000)
[Event #00000004] 0220 ( 00000000, 00000001, 00000001)
[Event #00000005] 0220 ( 00000000, 00000001, 00000002)
[Event #00000006] 0220 ( 00000000, 00000001, 00000003)
[Event #00000007] 0220 ( 00000000, 00000001, 00000004)
[Event #00000008] 0220 ( 00000000, 00000001, 00000005)
[Event #00000009] 0220 ( 00000000, 00000001, 00000006)
[Event #0000000A] 0220 ( 00000000, 00000001, 00000007)
[Event #0000000B] 0220 ( 00000000, 00000001, 00000008)
[Event #0000000C] 0220 ( 00000000, 00000001, 00000009)
[Event #0000000D] 0220 ( 00000000, 00000001, 0000000A)
[Event #0000000E] 0220 ( 00000000, 00000001, 0000000B)
[Event #0000000F] 0220 ( 00000000, 00000001, 0000000C)
.
.
.
[Event #000001B4] 0220 ( 00000000, 00000001, 000001B6)
[Event #000001B5] 0220 ( 00000000, 00000001, 000001B7)
[Event #000001B6] 0220 ( 00000000, 00000001, 000001B8)
[Event #000001B7] 0010 ( 00000000, 00000001 )

i'm going to try to actually rip one of these and see if i can get it repaired and up on the screen. hopefully i can figure it out.

i'll post as soon as i can get anything.

774

1st November 2008, 21:42

Did anyone ever find any use for PortableBDVM (http://bdvm.100webspace.net/)? I've been working on other projects so I haven't had any time to update it with all the discovered traps myself, but since it seems most of them have been figured out, maybe it'd be time to make it actually useful.

I'm still too busy to do it right now, but does anyone else feel like taking up the torch? If you want this code in mplayer or VLC or most any other media player, there needs to be a C version, so it could be a good place to start.

kkloster21

1st November 2008, 22:43

I successfully repaired and played Live Free or Die Hard! (I didn't play through the whole film but i watched enough to make sure that the content was repaired).

great job guys! hopefully KenD00 can implement this in dumpHD so that we don't have to rip the discs before we can play them :).

let me know what else i can do to help.

Oopho2ei

1st November 2008, 23:00

I'm still too busy to do it right now, but does anyone else feel like taking up the torch? If you want this code in mplayer or VLC or most any other media player, there needs to be a C version, so it could be a good place to start.
Accident has been working for some weeks on a C implementation now. Maybe you could both join your efforts. I've sent him a private message. Maybe he will contact you soon. Would be great. :)

I successfully repaired and played Live Free or Die Hard! (I didn't play through the whole film but i watched enough to make sure that the content was repaired).
Great. Could you also rip the disc with AnyDVD-HD and compare the files (with vbindiff, md5sum or whatever) please?

let me know what else i can do to help.
I need the conversion tables of those movies which are handled correctly (compared with AnyDVD-HD result) together with the movie title and the volume id. Also a list of movies for which you can't get the conversion table or for which the console output contains warnings or error messages.
:thanks:

KenD00

1st November 2008, 23:52

Success :).

Using the latest debugger i got a working conversion table (http://www.sendspace.com/file/xyr9fx) for Die Hard 4. I ripped the disc again with AnyDVD HD and the main movie file 00020.m2ts has in both cases the MD5 sum 0cbf12c33948e458883b060253f5f8d4.

While i made the old, non working, conversion table i made a mistake, i ran the debugger on the already aacs decrypted disc contents. I ran it again with the original disc and it produced a slightly different conversion table, about 20 spots in the files are different. However, using the new debugger i get identical conversion tables in both cases :confused:.

Now that i have a working conversion table i can start to implement support for this into DumpHD, but it will take some, i'm quite busy right now, so please be patient.

Oh, and hello arnezami, nice to see you again.

Edit:
Your BouncyCastle Debugger package doesn't work because its not enough to just put the jar archive into the classpath, you have also to register the provider. This can be done either statically by editing the java.security file or dynamically from the debugger. If you add this line to the debugger (make sure it gets executed only once)
java.security.Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
it will work (well, should, haven't tested this *g*).

:rolleyes:

Accident

2nd November 2008, 00:51

Accident has been working for some weeks on a C implementation now. Maybe you could both join your efforts. I've sent him a private message. Maybe he will contact you soon. Would be great. :)

I have finished the Slot code, but I need to make it right before it gets checked in (memory dumps differ, I suspect the flash.bin is actually little endian). After that, trap_PrivateKey, and events are left to be done. At that point it should be able to do what the Java code does.

In theory.

Oopho2ei

2nd November 2008, 01:13

Apparently there are no major issues so i think it's time for the initial release: BDVM Debugger v0.1.1 (http://uploaded.to/?id=sgx0g8)

The BouncyCastle security provider library is now included so at least in linux you don't need to install it manually (thanks to KenD00!) Simply call the debugger with "sh debugger.sh". Strangely the Debugger.jar is much larger now almost as if it includes the BouncyCastle library. But still the debugger fails to load if i remove the library from the classpath so you can find it in the lib/ directory after extraction. :)

Doom9

2nd November 2008, 12:48

@Oopho2ei: the jar file is so large because it contains the bouncycastle lib.. it's packed into the jar file. Something must've gone wrong during the build mechanism. And if you distribute the lib with the software you might want to build so that the lib in the dist directory is automatically taken into account (like Netbeans does).

Oopho2ei

2nd November 2008, 18:53

@Oopho2ei: the jar file is so large because it contains the bouncycastle lib.. it's packed into the jar file. Something must've gone wrong during the build mechanism. And if you distribute the lib with the software you might want to build so that the lib in the dist directory is automatically taken into account (like Netbeans does).
I have uploaded version 0.1.1 (see posting #327 (http://forum.doom9.org/showpost.php?p=1208974&postcount=327)) which hopefully fixed the BouncyCastle problem. Luckily i have found a working example build.xml for ant which includes the BouncyCastle security provider so i could fix this issue quickly. Even in windows or any other operating system you should be able to run the debugger by simply typing "java -jar <filename of the jar>" in the console. In linux you should use "sh debugger.sh" as described in the readme.

If everything works as it should you don't need to install BouncyCastle manually anymore because it is already included in the jar. :)

Accident

3rd November 2008, 01:56

Anyone here familiar with OpenSSL's ecdsa (http://www.openssl.org/docs/crypto/ecdsa.html)? I am unsure how to apply q,a,b,x,y and d with the OpenSSL API. Or if indeed those BigInts are standard, and I just need to set PrivateKey and sign?

lightshadow

3rd November 2008, 02:20

Now that you all have succeeded in hacking BD+ and released the source code, can any of you give a cookbook how to:

obtain the 7 keys
obtain memory footprints
obtain player signatures
obtain new traps
detect that BD+ developers have found a bug in the VM and uses that against us?

Apart from that the purpose of the program (called "content code") running inside the player on a virtual machine is to detect any known compromised players or known unlicensed emulators (like ours). The content code is give a wide range of opportunities to do that. For example it has (limited) access to the player memory and can even execute arbitrary code on the machine though we haven't seen that yet and our emulator doesn't support this either.
As long as we have access to a working (licensed) players all these measures are useless as we can record traces from this player and adjust the data "injected" in the virtual machine address space by traps or events to perfectly match our recordings. Even if whitebox attack resistant AES or ECDSA algorithms are used and nobody manages to break them we can still use the obfuscated algorithms and their keys.

Can a cookbook be given to this?

Also, can any of you show a diagram how BD+ works, like the famous AACS (http://www.blurayfreak.com/images/2007/04/16/aacs_highdef_decryption_diagram.jpg) diagram? :)

KenD00

3rd November 2008, 20:07

@schluppo
I'm trying to send you a PM but the forum says you either don't want or are not allowed to receive PM's :confused:. Can you check your settings please?

:rolleyes:

FoxDisc

3rd November 2008, 21:46

... what can they do to protect future releases? With AACS, it's easy, just revoke the keys used. Does your implementation of BD+ use any revokable keys, or are they limited to the unused instructions to try and fool you in the future?

The key set currently in use consists of 7 AES keys for TRAP_Aes and two ECDSA public/private key pairs for TRAP_PrivateKey. These keys can theoretically be revoked (e.g. by adding 7 more AES keys and don't use the old ones anymore for new movies)

Where are these keys located? For example, in AACS, the device stores device keys and the disc stores media keys and the VID. The media keys are easily readable and copyable, but they are encrypted. The VID is supposedly non-readable (outside of an authenticated AACS session) and is in a supposedly non-copyable area of the disc. You need the device keys to decrypt the media keys.

Where are the 7 AES keys and 2 ECDSA public/private key pairs kept - device or disc? Are they associated with a specific device, like the device keys, or do all devices have the same keys?

Doom9

4th November 2008, 09:30

@Oopho2ei: if you see this, could you check your PMs please?

Accident

5th November 2008, 08:03

I have yet to implement trap_PrivateKey, as the OpenSSL documentation is rather sparse. So for now, it is returning static bytes as per snapshot, and completes the run until the end:

-rw-r--r-- 1 owner group 3211264 Nov 5 15:47 conv_tab.bin

00000000 00 24 00 00 00 01 01 53 00 00 05 54 00 00 05 58 |.$.....S...T...X|
00000010 00 00 0f 84 00 00 1f c8 00 00 2e 54 00 00 3c f4 |.......<C8>...T..|

I do not know if it is entirely correct as I do not have the media itself, but at least it starts with 0x24/36 segments, when it is ready.

libbluray is written in C, uses autoconf and has a MSVC++ project file, and is endian aware, so it should work on all platforms.

Edit1:
The fastest run I have managed to do takes 4 seconds. On average, up to 15 seconds. This is with loading cores for Random and PrivateKey. (If I use my Random, it fails, probably because PrivateKey is not yet implemented). I haven't really looked at optimisation yet.

tteich

5th November 2008, 09:05

I have yet to implement trap_PrivateKey, as the OpenSSL documentation is rather sparse. So for now, it is returning static bytes as per snapshot, and completes the run until the end:

-rw-r--r-- 1 owner group 3211264 Nov 5 15:47 conv_tab.bin

00000000 00 24 00 00 00 01 01 53 00 00 05 54 00 00 05 58 |.$.....S...T...X|
00000010 00 00 0f 84 00 00 1f c8 00 00 2e 54 00 00 3c f4 |.......<C8>...T..|

I do not know if it is entirely correct as I do not have the media itself, but at least it starts with 0x24/36 segments, when it is ready.

libbluray is written in C, uses autoconf and has a MSVC++ project file, and is endian aware, so it should work on all platforms.
That's great, what you're contributing! A native implementation in C opens doors to have an open source BD player sometimes later...

lightshadow

5th November 2008, 14:33

I have yet to implement trap_PrivateKey, as the OpenSSL documentation is rather sparse.

Have you tried the openSSL mailinglist (http://openssl.org/support/)? It is fairly high trafic, but have a high signal to noise ratio.

I wonder if the GPLv3 allows DRM breaking tools :)

dirio49

5th November 2008, 22:11

Does anybody think this thread should be a sticky? :)

LoRd_MuldeR

6th November 2008, 00:37

I wonder if the GPLv3 allows DRM breaking tools :)

Why not? The GPL defines what other people are allowed/required to do with your work. It doesn't put any restrictions on your work itself.

Also the basic idea of the GPL is "Copyleft" - keep free stuff free. Hence healing DRM-infected files corresponds to the "spirit" of the GPL.

XAvAX

6th November 2008, 03:55

Does anybody think this thread should be a sticky? :)

Seconded.

Accident

7th November 2008, 00:08

I have been working on trap_PrivateKey() and I must say I was close, but in the end Oopho2ei provided the working version. All traps are now handled without loading cores, and a conv_tab.bin file is created.

# time ./test/bdtest -t -1
real 0m0.917s
user 0m0.767s
sys 0m0.031s
# ls -l dat/
-rw-r--r-- 1 owner group 3211264 Nov 7 08:03 conv_tab.bin

The tab files differ, so there will most likely be more things to fix first.

Edit:
Oh heh, further clarification is that the file is still encoded, so naturally it differs.

Rupan

7th November 2008, 03:27

Congratulations to Oopho2ei, Accident, and all the others whose hard work has made this possible. I think it might be a good idea at this point to post instructions for playing video in some F/OSS player. Here is my solution for mplayer:

1) Add this to /etc/sysctl.conf:

dev.rtc.max-user-freq = 1024

2) Add this to ~/.mplayer/config:

[bluray]
profile-desc="Blu-Ray mpeg2 transport stream"
demuxer=lavf
fps=24000/1001
fs=true
lavdopts=threads=2:fast=true:skiploopfilter=all

3) Play your video:

mplayer -profile bluray -cache 50000 000000.m2ts

Accident

10th November 2008, 14:05

I have checked in a version that sends events for each Segment that needs it, and attempts to decode the subtables and segments. I am not entirely convinced that part is quite correct however. There most likely are a lot of other bugs too. But for now, it just about makes _something_ you can run.

For those wanting to try a tarball, download it directly instead.libbluray-0.0.1.tar.gz (http://uploaded.to/?id=j5siat)

Currently takes about 1.3seconds to run on my machine.

# time ./test/bdtest -t -1
real 0m1.324s
user 0m1.293s
sys 0m0.017s
# ls -l dat/conv_tab.bin
-rw-r--r-- 1 owner group 1035678 Nov 10 22:00 dat/conv_tab.bin

00000000 00 24 00 00 00 01 01 53 00 00 05 54 00 00 05 58
00000010 00 00 0f 84 00 00 1f c8 00 00 2e 54 00 00 3c f4

00000550 00 0f c1 e4 00 00 00 00 00 00 00 82 00 09 90 0a
00000560 fc d4 0a fd 56 7f 9a ff fc 4b 51 3f 99 fa 69 92

Edit:
Ah no, definitely wrong, I do not save the 2 * 12 bits address-adjustment fields to the file correctly.

Edit2:

As a real-life example, I tested the libbluray code on a Sigma SMP8634 board (MIPS 4KEc 300MHz CPU)using the DAT BD+ files provided here. Currently execution takes 1:09 to produce the conv_tab.bin file. (No precise timing, it does not have "time".). Perhaps a little optimisation is in order, as who wants to wait a full minute to watch their movie.

libbluray-0.0.1 # uname -a
Linux PCH-A100 2.6.15-sigma #2251 PREEMPT Thu Oct 23 18:44:01 MYT 2008 mips unknown

libbluray-0.0.1 # date ; test/bdtest -t -1 ; date
Tue Nov 11 10:45:25 GMT 2008
Tue Nov 11 10:46:34 GMT 2008

libbluray-0.0.1 # ls -l dat
-rw-r--r-- 1 root root 1035678 Nov 11 10:46 conv_tab.bin

loric

11th November 2008, 14:05

As a real-life example, I tested the libbluray code on a Sigma SMP8634 board (MIPS 4KEc 300MHz CPU)using the DAT BD+ files provided here. Currently execution takes 1:09 to produce the conv_tab.bin file. (No precise timing, it does not have "time".). Perhaps a little optimisation is in order, as who wants to wait a full minute to watch their movie.

The same problem affects many stand-alone BD players: after you hit play and before you can actually start watching a BD movie, protected with BD+, you have all the time to prepare a coffee espresso, drink it and wash the cup.

How fast is that on an average PC?

Accident

11th November 2008, 14:15

Just under a second. You can see post #341.

loric

11th November 2008, 14:34

Just under a second. You can see post #341.

Sorry, I had missed that.

Anyway, I think the typical use for your library would be a HTPC, and 1 second is fine :)

Accident

12th November 2008, 09:12

I committed the latest version which includes some bugs found in the conv_tab.bin file. I can now confirm the file is identical to the file produced by the Debugger.

I made an additional test program, which is a little friendlier to produce the conv_tab.bin file than the previous VM simulator tool.

I have also created Windows VC 2008 project files, and the tarball has compiled binary executables for Windows if people want to do a quick test. My Windows laptop takes 12.7s with Debug build, and 6.9seconds with Release build.

libbluray-0.0.2.tar.gz (http://uploaded.to/?id=536jqp)

DAT files:

# time test/convtab -d dat/ -I 1d2fc67462fdf986357a8f808fa1298b -f dat/flash.bin

real 0m1.317s
user 0m1.270s
sys 0m0.019s

Edit:

I added the ability to patch files using the obtained conv_tab.bin file. I do not actually have DAT, but using mkfile, I can confirm it appears to do the right thing.

segment: direct patch title 0 started.
[segment] index 9900A
[segment] flags 4A
[segment] adjust0 01AA
[segment] adjust1 0053
[segment] offset0 64
[segment] offset1 B1
[segment] patch0 AA2D408E4A
[segment] patch1 4DEF4B6BE5
[segment] would seek to 00000000072d4764 to write patch0
[segment] would seek to 00000000072d85f1 to write patch1
[segment] patching completed.
real 20m24.281s

# hexdump -C 00001.m2ts |less
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
072d4760 00 00 00 00 aa 2d 40 8e 4a 00 00 00 00 00 00 00 |....<AJ.......|

-rw------- 1 owner group 31948116575 Nov 13 15:23 00001.m2ts

I will also make a buffering style syntax, so you can do something along the lines of

aacsdecode 00001.m2ts | convtab -t 1 -U | mplayer

Since the primary focus of the C libbluray was to be a library used by other software.

tteich

12th November 2008, 15:48

Just under a second. You can see post #341.
I think the native code can be improved further, if needed. For the time being the performance you reported seems to be fine. You can decode the audio and video parts of an m2ts in real time on a decent processor, so adding a few sec for BD+ removal seems to be not a problem.

loo3aem3ON

12th November 2008, 16:20

Where are the 7 AES keys and 2 ECDSA public/private key pairs kept - device or disc? Are they associated with a specific device, like the device keys, or do all devices have the same keys?
All devices have a ECDSA public/private key pair as well as these seven AES keys stored (maybe in obfuscated form) in their memory. The content code contains some public keys to e.g. verify the signature of the certificates which the player presents with TRAP_DeviceDiscovery. The player specific AES keys can for example be used to decrypt content code designed for a specific player. Other AES keys are common among a small group of players.

Some people have started writing a documentation which you can find here (http://uploaded.to/?id=c3sl4m) (latex source (http://trac.assembla.com/bdplus/browser/trunk/bdplusintro)). It's currently very sparse but you can still get an impression of how BD+ works.

xkodi

12th November 2008, 19:42

just to report that it doesn't work with Ice Age, both "BDVM Debugger v0.1.1" and libbluray-0.0.2 produce the same byte by byte identical conv_tab.bin, but the reconstructed M2TS file with content_repair.c is even more broken than the original BD+ protected one.

loo3aem3ON

12th November 2008, 19:50

just to report that it doesn't work with Ice Age, both "BDVM Debugger v0.1.1" and libbluray-0.0.2 produce the same byte by byte identical conv_tab.bin, but the reconstructed M2TS file with content_repair.c is even more broken than the BD+ protected one.
Please upload the conv_tab.bin (uploaded.to/rapidshare/whatever..). Also try to use DumpHD v0.5 which supports repairing BD+ protected movies with a given decrypted conversion table.

Thanks for reporting.

KenD00

12th November 2008, 20:14

Have you opened the conv_tab.bin with ConvTableView 0.2? Does the log show any messages?

:rolleyes:

xkodi

12th November 2008, 22:26

Please upload the conv_tab.bin (uploaded.to/rapidshare/whatever..).

http://rapidshare.de/files/40895949/conv_tab.bin.html

Have you opened the conv_tab.bin with ConvTableView 0.2? Does the log show any messages?

yes, i can see the segments with ConvTableView 0.2 without any errors, also no errors are reported by the BDVM Debugger:

java -jar bdvmdbg-0.1.1.jar "$@"
[W] No post-trap snapshot archive found!
[W] No post-break snapshot archive found!
[W] No program counter trace found!
[W] No instruction trace found!
[W] No timer trace found!
Loading /iceage/BDSVM/00000.svm ...
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[Event #00000002] 0110 ( 00000000, 00000001 )
Conversion table set
[Event #00000003] 0220 ( 00000000, 00000001, 00000000)
[Event #00000004] 0220 ( 00000000, 00000001, 00000001)
[Event #00000005] 0220 ( 00000000, 00000001, 00000002)
[Event #00000006] 0220 ( 00000000, 00000001, 00000003)
[Event #00000007] 0220 ( 00000000, 00000001, 00000004)
[Event #00000008] 0220 ( 00000000, 00000001, 00000005)
[Event #00000009] 0220 ( 00000000, 00000001, 00000006)
[Event #0000000A] 0220 ( 00000000, 00000001, 00000007)
[Event #0000000B] 0220 ( 00000000, 00000001, 00000008)
[Event #0000000C] 0220 ( 00000000, 00000001, 00000009)
[Event #0000000D] 0220 ( 00000000, 00000001, 0000000A)
[Event #0000000E] 0220 ( 00000000, 00000001, 0000000B)
[Event #0000000F] 0220 ( 00000000, 00000001, 0000000C)
[Event #00000010] 0220 ( 00000000, 00000001, 0000000D)
[Event #00000011] 0220 ( 00000000, 00000001, 0000000E)
[Event #00000012] 0220 ( 00000000, 00000001, 0000000F)
[Event #00000013] 0220 ( 00000000, 00000001, 00000010)
[Event #00000014] 0220 ( 00000000, 00000001, 00000011)
[Event #00000015] 0220 ( 00000000, 00000001, 00000012)
[Event #00000016] 0220 ( 00000000, 00000001, 00000013)
[Event #00000017] 0220 ( 00000000, 00000001, 00000014)
[Event #00000018] 0220 ( 00000000, 00000001, 00000015)
[Event #00000019] 0220 ( 00000000, 00000001, 00000016)
[Event #0000001A] 0220 ( 00000000, 00000001, 00000017)
[Event #0000001B] 0220 ( 00000000, 00000001, 00000018)
[Event #0000001C] 0220 ( 00000000, 00000001, 00000019)
[Event #0000001D] 0220 ( 00000000, 00000001, 0000001A)
[Event #0000001E] 0220 ( 00000000, 00000001, 0000001B)
[Event #0000001F] 0220 ( 00000000, 00000001, 0000001C)
[Event #00000020] 0220 ( 00000000, 00000001, 0000001D)
[Event #00000021] 0220 ( 00000000, 00000001, 0000001E)
[Event #00000022] 0220 ( 00000000, 00000001, 0000001F)
[Event #00000023] 0220 ( 00000000, 00000001, 00000020)
[Event #00000024] 0220 ( 00000000, 00000001, 00000021)
[Event #00000025] 0220 ( 00000000, 00000001, 00000022)
[Event #00000026] 0220 ( 00000000, 00000001, 00000023)
[Event #00000027] 0220 ( 00000000, 00000001, 00000024)
[Event #00000028] 0220 ( 00000000, 00000001, 00000025)
[Event #00000029] 0220 ( 00000000, 00000001, 00000026)
[Event #0000002A] 0220 ( 00000000, 00000001, 00000027)
[Event #0000002B] 0220 ( 00000000, 00000001, 00000028)
[Event #0000002C] 0220 ( 00000000, 00000001, 00000029)
[Event #0000002D] 0220 ( 00000000, 00000001, 0000002A)
[Event #0000002E] 0220 ( 00000000, 00000001, 0000002B)
[Event #0000002F] 0220 ( 00000000, 00000001, 0000002C)
[Event #00000030] 0220 ( 00000000, 00000001, 0000002D)
[Event #00000031] 0220 ( 00000000, 00000001, 0000002E)
[Event #00000032] 0220 ( 00000000, 00000001, 0000002F)
[Event #00000033] 0220 ( 00000000, 00000001, 00000030)
[Event #00000034] 0220 ( 00000000, 00000001, 00000031)
[Event #00000035] 0220 ( 00000000, 00000001, 00000032)
[Event #00000036] 0220 ( 00000000, 00000001, 00000033)
[Event #00000037] 0220 ( 00000000, 00000001, 00000034)
[Event #00000038] 0220 ( 00000000, 00000001, 00000035)
[Event #00000039] 0220 ( 00000000, 00000001, 00000036)
[Event #0000003A] 0220 ( 00000000, 00000001, 00000037)
[Event #0000003B] 0220 ( 00000000, 00000001, 00000038)
[Event #0000003C] 0220 ( 00000000, 00000001, 00000039)
[Event #0000003D] 0220 ( 00000000, 00000001, 0000003A)
[Event #0000003E] 0220 ( 00000000, 00000001, 0000003B)
[Event #0000003F] 0220 ( 00000000, 00000001, 0000003C)
[Event #00000040] 0220 ( 00000000, 00000001, 0000003D)
[Event #00000041] 0220 ( 00000000, 00000001, 0000003E)
[Event #00000042] 0220 ( 00000000, 00000001, 0000003F)
[Event #00000043] 0220 ( 00000000, 00000001, 00000040)
[Event #00000044] 0220 ( 00000000, 00000001, 00000041)
[Event #00000045] 0220 ( 00000000, 00000001, 00000042)
[Event #00000046] 0220 ( 00000000, 00000001, 00000043)
[Event #00000047] 0220 ( 00000000, 00000001, 00000044)
[Event #00000048] 0220 ( 00000000, 00000001, 00000045)
[Event #00000049] 0220 ( 00000000, 00000001, 00000046)
[Event #0000004A] 0220 ( 00000000, 00000001, 00000047)
[Event #0000004B] 0220 ( 00000000, 00000001, 00000048)
[Event #0000004C] 0220 ( 00000000, 00000001, 00000049)
[Event #0000004D] 0220 ( 00000000, 00000001, 0000004A)
[Event #0000004E] 0220 ( 00000000, 00000001, 0000004B)
[Event #0000004F] 0220 ( 00000000, 00000001, 0000004C)
[Event #00000050] 0220 ( 00000000, 00000001, 0000004D)
[Event #00000051] 0220 ( 00000000, 00000001, 0000004E)
[Event #00000052] 0220 ( 00000000, 00000001, 0000004F)
[Event #00000053] 0220 ( 00000000, 00000001, 00000050)
[Event #00000054] 0220 ( 00000000, 00000001, 00000051)
[Event #00000055] 0220 ( 00000000, 00000001, 00000052)
[Event #00000056] 0220 ( 00000000, 00000001, 00000053)
[Event #00000057] 0220 ( 00000000, 00000001, 00000054)
[Event #00000058] 0220 ( 00000000, 00000001, 00000055)
[Event #00000059] 0220 ( 00000000, 00000001, 00000056)
[Event #0000005A] 0220 ( 00000000, 00000001, 00000057)
[Event #0000005B] 0220 ( 00000000, 00000001, 00000058)
[Event #0000005C] 0220 ( 00000000, 00000001, 00000059)
[Event #0000005D] 0220 ( 00000000, 00000001, 0000005A)
[Event #0000005E] 0220 ( 00000000, 00000001, 0000005B)
[Event #0000005F] 0220 ( 00000000, 00000001, 0000005C)
[Event #00000060] 0220 ( 00000000, 00000001, 0000005D)
[Event #00000061] 0220 ( 00000000, 00000001, 0000005E)
[Event #00000062] 0220 ( 00000000, 00000001, 0000005F)
[Event #00000063] 0220 ( 00000000, 00000001, 00000060)
[Event #00000064] 0220 ( 00000000, 00000001, 00000061)
[Event #00000065] 0220 ( 00000000, 00000001, 00000062)
[Event #00000066] 0220 ( 00000000, 00000001, 00000063)
[Event #00000067] 0220 ( 00000000, 00000001, 00000064)
[Event #00000068] 0220 ( 00000000, 00000001, 00000065)
[Event #00000069] 0220 ( 00000000, 00000001, 00000066)
[Event #0000006A] 0220 ( 00000000, 00000001, 00000067)
[Event #0000006B] 0220 ( 00000000, 00000001, 00000068)
[Event #0000006C] 0220 ( 00000000, 00000001, 00000069)
[Event #0000006D] 0220 ( 00000000, 00000001, 0000006A)
[Event #0000006E] 0220 ( 00000000, 00000001, 0000006B)
[Event #0000006F] 0220 ( 00000000, 00000001, 0000006C)
[Event #00000070] 0220 ( 00000000, 00000001, 0000006D)
[Event #00000071] 0220 ( 00000000, 00000001, 0000006E)
[Event #00000072] 0220 ( 00000000, 00000001, 0000006F)
[Event #00000073] 0220 ( 00000000, 00000001, 00000070)
[Event #00000074] 0220 ( 00000000, 00000001, 00000071)
[Event #00000075] 0220 ( 00000000, 00000001, 00000072)
[Event #00000076] 0220 ( 00000000, 00000001, 00000073)
[Event #00000077] 0220 ( 00000000, 00000001, 00000074)
[Event #00000078] 0220 ( 00000000, 00000001, 00000075)
[Event #00000079] 0220 ( 00000000, 00000001, 00000076)
[Event #0000007A] 0220 ( 00000000, 00000001, 00000077)
[Event #0000007B] 0220 ( 00000000, 00000001, 00000078)
[Event #0000007C] 0220 ( 00000000, 00000001, 00000079)
[Event #0000007D] 0220 ( 00000000, 00000001, 0000007A)
[Event #0000007E] 0220 ( 00000000, 00000001, 0000007B)
[Event #0000007F] 0220 ( 00000000, 00000001, 0000007C)
[Event #00000080] 0220 ( 00000000, 00000001, 0000007D)
[Event #00000081] 0220 ( 00000000, 00000001, 0000007E)
[Event #00000082] 0220 ( 00000000, 00000001, 0000007F)
[Event #00000083] 0220 ( 00000000, 00000001, 00000080)
[Event #00000084] 0220 ( 00000000, 00000001, 00000081)
[Event #00000085] 0220 ( 00000000, 00000001, 00000082)
[Event #00000086] 0220 ( 00000000, 00000001, 00000083)
[Event #00000087] 0220 ( 00000000, 00000001, 00000084)
[Event #00000088] 0220 ( 00000000, 00000001, 00000085)
[Event #00000089] 0220 ( 00000000, 00000001, 00000086)
[Event #0000008A] 0220 ( 00000000, 00000001, 00000087)
[Event #0000008B] 0220 ( 00000000, 00000001, 00000088)
[Event #0000008C] 0220 ( 00000000, 00000001, 00000089)
[Event #0000008D] 0220 ( 00000000, 00000001, 0000008A)
[Event #0000008E] 0220 ( 00000000, 00000001, 0000008B)
[Event #0000008F] 0220 ( 00000000, 00000001, 0000008C)
[Event #00000090] 0220 ( 00000000, 00000001, 0000008D)
[Event #00000091] 0220 ( 00000000, 00000001, 0000008E)
[Event #00000092] 0220 ( 00000000, 00000001, 0000008F)
[Event #00000093] 0220 ( 00000000, 00000001, 00000090)
[Event #00000094] 0220 ( 00000000, 00000001, 00000091)
[Event #00000095] 0220 ( 00000000, 00000001, 00000092)
[Event #00000096] 0220 ( 00000000, 00000001, 00000093)
[Event #00000097] 0220 ( 00000000, 00000001, 00000094)
[Event #00000098] 0220 ( 00000000, 00000001, 00000095)
[Event #00000099] 0220 ( 00000000, 00000001, 00000096)
[Event #0000009A] 0220 ( 00000000, 00000001, 00000097)
[Event #0000009B] 0220 ( 00000000, 00000001, 00000098)
[Event #0000009C] 0220 ( 00000000, 00000001, 00000099)
[Event #0000009D] 0220 ( 00000000, 00000001, 0000009A)
[Event #0000009E] 0220 ( 00000000, 00000001, 0000009B)
[Event #0000009F] 0220 ( 00000000, 00000001, 0000009C)
[Event #000000A0] 0220 ( 00000000, 00000001, 0000009D)
[Event #000000A1] 0220 ( 00000000, 00000001, 0000009E)
[Event #000000A2] 0220 ( 00000000, 00000001, 0000009F)
[Event #000000A3] 0220 ( 00000000, 00000001, 000000A0)
[Event #000000A4] 0220 ( 00000000, 00000001, 000000A1)
[Event #000000A5] 0220 ( 00000000, 00000001, 000000A2)
[Event #000000A6] 0220 ( 00000000, 00000001, 000000A3)
[Event #000000A7] 0220 ( 00000000, 00000001, 000000A4)
[Event #000000A8] 0220 ( 00000000, 00000001, 000000A5)
[Event #000000A9] 0220 ( 00000000, 00000001, 000000A6)
[Event #000000AA] 0220 ( 00000000, 00000001, 000000A7)
[Event #000000AB] 0220 ( 00000000, 00000001, 000000A8)
[Event #000000AC] 0220 ( 00000000, 00000001, 000000A9)
[Event #000000AD] 0220 ( 00000000, 00000001, 000000AA)
[Event #000000AE] 0220 ( 00000000, 00000001, 000000AB)
[Event #000000AF] 0220 ( 00000000, 00000001, 000000AC)
[Event #000000B0] 0220 ( 00000000, 00000001, 000000AD)
[Event #000000B1] 0220 ( 00000000, 00000001, 000000AE)
[Event #000000B2] 0220 ( 00000000, 00000001, 000000AF)
[Event #000000B3] 0220 ( 00000000, 00000001, 000000B0)
[Event #000000B4] 0220 ( 00000000, 00000001, 000000B1)
[Event #000000B5] 0220 ( 00000000, 00000001, 000000B2)
[Event #000000B6] 0220 ( 00000000, 00000001, 000000B3)
[Event #000000B7] 0220 ( 00000000, 00000001, 000000B4)
[Event #000000B8] 0220 ( 00000000, 00000001, 000000B5)
[Event #000000B9] 0220 ( 00000000, 00000001, 000000B6)
[Event #000000BA] 0220 ( 00000000, 00000001, 000000B7)
[Event #000000BB] 0220 ( 00000000, 00000001, 000000B8)
[Event #000000BC] 0220 ( 00000000, 00000001, 000000B9)
[Event #000000BD] 0220 ( 00000000, 00000001, 000000BA)
[Event #000000BE] 0220 ( 00000000, 00000001, 000000BB)
[Event #000000BF] 0220 ( 00000000, 00000001, 000000BC)
[Event #000000C0] 0220 ( 00000000, 00000001, 000000BD)
[Event #000000C1] 0220 ( 00000000, 00000001, 000000BE)
[Event #000000C2] 0220 ( 00000000, 00000001, 000000BF)
[Event #000000C3] 0220 ( 00000000, 00000001, 000000C0)
[Event #000000C4] 0220 ( 00000000, 00000001, 000000C1)
[Event #000000C5] 0220 ( 00000000, 00000001, 000000C2)
[Event #000000C6] 0220 ( 00000000, 00000001, 000000C3)
[Event #000000C7] 0220 ( 00000000, 00000001, 000000C4)
[Event #000000C8] 0220 ( 00000000, 00000001, 000000C5)
[Event #000000C9] 0220 ( 00000000, 00000001, 000000C6)
[Event #000000CA] 0220 ( 00000000, 00000001, 000000C7)
[Event #000000CB] 0220 ( 00000000, 00000001, 000000C8)
[Event #000000CC] 0220 ( 00000000, 00000001, 000000C9)
[Event #000000CD] 0220 ( 00000000, 00000001, 000000CA)
[Event #000000CE] 0220 ( 00000000, 00000001, 000000CB)
[Event #000000CF] 0220 ( 00000000, 00000001, 000000CC)
[Event #000000D0] 0220 ( 00000000, 00000001, 000000CD)
[Event #000000D1] 0220 ( 00000000, 00000001, 000000CE)
[Event #000000D2] 0220 ( 00000000, 00000001, 000000CF)
[Event #000000D3] 0220 ( 00000000, 00000001, 000000D0)
[Event #000000D4] 0220 ( 00000000, 00000001, 000000D1)
[Event #000000D5] 0220 ( 00000000, 00000001, 000000D2)
[Event #000000D6] 0220 ( 00000000, 00000001, 000000D3)
[Event #000000D7] 0220 ( 00000000, 00000001, 000000D4)
[Event #000000D8] 0220 ( 00000000, 00000001, 000000D5)
[Event #000000D9] 0220 ( 00000000, 00000001, 000000D6)
[Event #000000DA] 0220 ( 00000000, 00000001, 000000D7)
[Event #000000DB] 0220 ( 00000000, 00000001, 000000D8)
[Event #000000DC] 0220 ( 00000000, 00000001, 000000D9)
[Event #000000DD] 0220 ( 00000000, 00000001, 000000DA)
[Event #000000DE] 0220 ( 00000000, 00000001, 000000DB)
[Event #000000DF] 0220 ( 00000000, 00000001, 000000DC)
[Event #000000E0] 0220 ( 00000000, 00000001, 000000DD)
[Event #000000E1] 0220 ( 00000000, 00000001, 000000DE)
[Event #000000E2] 0220 ( 00000000, 00000001, 000000DF)
[Event #000000E3] 0220 ( 00000000, 00000001, 000000E0)
[Event #000000E4] 0220 ( 00000000, 00000001, 000000E1)
[Event #000000E5] 0220 ( 00000000, 00000001, 000000E2)
[Event #000000E6] 0010 ( 00000000, 00000001 )

loo3aem3ON

12th November 2008, 23:35

Thanks. The conversion table looks correct but i can't tell for sure. I noticed that you haven't manually set the volume id. Did you modify the volume_id.bin to match your volume id?

My current guess is the conversion table wasn't correctly applied to the movie. Could you please try to run dumpHD like this (thanks to kkloster21):
/dumphd-0.5/dumphd.sh --infile:BDMV/STREAM/00001.m2ts --convtable:conv_tab.bin /media/cdrom/ > /tmp/00001.m2ts
Also keep in mind that once you apply a conversion table to the *.m2ts files they are modified irreversibly. For instance if you made a mistake adapting the content_repair.c to your movie the *.m2ts files are broken and they cannot be repaired even if you subsequently apply the correct conversion table in a correct manner. You have to get a clean copy of the original (AACS decrypted) m2ts files and try again.:rolleyes:

xkodi

13th November 2008, 01:16

Thanks. The conversion table looks correct but i can't tell for sure. I noticed that you haven't manually set the volume id. Did you modify the volume_id.bin to match your volume id?

no, matter what i set for volume id the conv_tab.bin is always the same. i tried with Die Hard 4 and even with wrong volume id the produced conv_tab.bin is correct one - the same as the one posted earlier in this thread. so, it seems that volume id is not used at all !?

if you made a mistake adapting the content_repair.c to your movie

i think that changes to content_repair.c are not needed, because Ice Age uses table 1 and 00001.m2ts, which are the same as The Day after Tomorrow. so, i don't expect using dumphd to make any difference.

maybe, i'm doing something wrong and hope that someone else will try with Ice Age and report back too.

MickJT

13th November 2008, 07:58

no, matter what i set for volume id the conv_tab.bin is always the same. i tried with Die Hard 4 and even with wrong volume id the produced conv_tab.bin is correct one - the same as the one posted earlier in this thread. so, it seems that volume id is not used at all !?

i think that changes to content_repair.c are not needed, because Ice Age uses table 1 and 00001.m2ts, which are the same as The Day after Tomorrow. so, i don't expect using dumphd to make any difference.

maybe, i'm doing something wrong and hope that someone else will try with Ice Age and report back too.
Which version of Ice Age do you have, and is it Ice Age or Ice Age 2? Which region? Or more specifically what's the Disc ID for it?

Here in Australia:

Unit Key File Hash (Disc ID): 655E67484F36F5734ED4BDDBA3FBC2CAA938ECEB

I used DumpVID to get the VID and then aacskeys to do the rest.

Edit: Furthermore, convtab and convtabv crashes on this disc (at least under Vista) and no .bin file is made. How peculiar. Should I supply a log in verbose mode?
Edit2: bdvm debugger 0.1.1 works fine however, and using the conv_tab.bin in DumpHD works perfectly and produces a m2ts file with BD+ properly removed... Also under Vista. (AnyDVD HD not running obviously).. tested without using conv_tab.bin and BD+ is indeed present and glitchy as always.

loric

14th November 2008, 16:37

Looks like there's a new version of BD+ out there, one that Slysoft's AnyDVD-HD can't circumvent as of now:
http://forum.slysoft.com/showthread.php?t=21985

It would be interesting to know how well this open project library can cope with it.

Doom9

14th November 2008, 18:40

Speaking of BD+ updates, I compiled a little list on Slysoft's BD+ updates and affected discs by the update - so if you're looking at validating the software against more advanced titles this should provide useful:

October 31st 6.4.7.9 (die another day, the happening, possibly: firefly/bender's game) - Slysoft now says they need 1-2 months to take care of those
August 22, 6.4.6.2 (Prison Break S03 US, Street Kings US)
August 1st, 6.4.5.9 (Jumper DE/UK release)
June 17th, 6.4.5.0 (Jumpger US release)
March 27 6.4.0.4 (Hitman, US release)

loo3aem3ON

14th November 2008, 19:52

The debugger can't handle jumper and probably most recently released titles. The implementation of TRAP_DiscoveryRAM has changed. A new second certificate for TRAP_DeviceDiscovery is used (consequently a new public/private ECDSA key) and the version key (aes key number 6) was updated as well. Furthermore there are bugs in TRAP_Sha and TRAP_MediaSHAFileHash which didn't show up before. The libblueray library from Accident might not have these problems because it's a different implementation.
I saw "the happening" in stores lately but the movie is awful so i didn't buy it.

loric

14th November 2008, 20:26

The debugger can't handle jumper and probably most recently released titles. The implementation of TRAP_DiscoveryRAM has changed. A new second certificate for TRAP_DeviceDiscovery is used (consequently a new public/private ECDSA key) and the version key (aes key number 6) was updated as well.

What happens with people that bought a stand-alone player? Do they need to update the firmware to play discs protected with the latest BD+ scheme? Or does that work like AASC, i.e. with revocations affecting only software players and, maybe, the PS3?

loo3aem3ON

14th November 2008, 20:56

What happens with people that bought a stand-alone player? Do they need to update the firmware to play discs protected with the latest BD+ scheme? Or does that work like AASC, i.e. with revocations affecting only software players and, maybe, the PS3?
They might have to update the player firmware by allowing the player to temporary access the internet or downloading the update manually and burning it to disc. Note that the content code is selecting which program to run based in it's analysis of TRAP_DeviceDiscovery, TRAP_DiscoveryRAM etc ("fingerprint"). It can skip any intensive testing if the player is considered trustworthy (because no hacks known) and therefor it is unlikely that this player would need regular updates if any.
Furthermore don't think of it like the BD+ specification is constantly changing and everyone has to flash it's firmware every month or so. As usual the vendors haven't implemented every detail so their implementations are not fully BD+ conform but these players will probably work fine for a long period of time. This is changing once hacks for a particular player model are known. In this case the content code starts running a lot of checks and might even refuse to generate the conversion table if the certificates and other keys are too old. This is when the vendor is forced to release updates. I believe it is known to the industry which player AnyDVD-HD is emulating but SlySoft doesn't seem to be willing to switch to a different model. :rolleyes:

Slysoft now says they need 1-2 months to take care of those
That would be longer than we needed to reverse engineer the entire thing. strange...

tteich

14th November 2008, 21:37

[...]
That would be longer than we needed to reverse engineer the entire thing. strange...
Fox released quite a bit of titles, perhaps with different varieties of BD+ code. Should take a while to analyze and circumvent it. Perhaps we should try to get some of those advanced BD+ "debugged". I think the .zip files which user post on the slysoft forum contain the necessary bdvm files from the related disc.

Edit: just checked and realized that the zip file doesn't contain the /BDSVM contents :-(

loo3aem3ON

14th November 2008, 22:22

Edit: just checked and realized that the zip file doesn't contain the /BDSVM contents :-(
I have snapshots for jumper and i am optimistic to get this to work without the help of the snapshots as well. Be patient.

Accident

15th November 2008, 00:11

I would be interested in receiving content files for releases that are more advanced, or at least hear how it runs.

Accident

15th November 2008, 07:24

I have just checked-in version 0.0.3, which adds a streaming method to using the conv_tab.bin. For those wanted to include it is their own code, this is the pseudo code required.

bluray_init();
vm = dlx_VM_new(DLX_MEMORY_SIZE);
dlx_load_slots(vm, opt_flashfile ? opt_flashfile : "flash.bin");
dlx_setVolumeID(vm, volumeid);
dlx_loadsvm(vm, 0x0, "BDSVM/00000.svm");
dlx_setPC(vm, 0x1000);
dlx_setWD(vm, 0x7fffffff);
dlx_run_convtab(vm);
dlx_patchseek(vm, 1/*title*/, 0x00000000 /*seek*/);
while((rbytes = fread(buffer, 1,
sizeof(buffer),
stdin)) > 0) {
dlx_patch(vm, rbytes, buffer);
fwrite(buffer, rbytes, 1, stdout);
}

I have included Windows binaries again.

Download here:
http://uploaded.to/?id=ge5h4v

Unix style use are:

# ./configure
# make
# mount /dev/cdrom /mnt
# cat /mnt/BDMV/STREAM/00001.m2ts | test/convtab -d /mnt/BDMV -t 1 -U | mplayer
(Use -f and -i or -I to specify flash file, and VolumeID)

MickJT

15th November 2008, 14:12

convtab(d).exe still crashes on the Ice Age 1 Blu-Ray (Australian). Haven't checked under Linux.

Should I upload the contents of BDSVM somewhere? (excluding the BACKUP folder).

Edit: Other discs are fine.. It's just this one, and the bdvm debugger .jar works fine with it.
Edit2: I have read this thread thoroughly, but i'm unaware as to what flash.bin is used for, how it is obtained/created and where it's needed, if needed. I haven't needed it yet and BD+ is removed without any problems.

saint-francis

15th November 2008, 15:00

October 31st 6.4.7.9 (die another day, the happening, possibly: firefly/bender's game) - Slysoft now says they need 1-2 months to take care of those

I ripped the happening last night with no problem at all using anydvd. And I watched it from my HDD. Am I missing something here?

dirio49

15th November 2008, 16:15

I ripped the happening last night with no problem at all using anydvd. And I watched it from my HDD. Am I missing something here?

Please read the thread. It has nothing to do with anydvd. it is about opensource BD+ implementation.
Later

Edit:
James for anydvd says that they need at least one month. This is for Blu-ray not dvds. :)

saint-francis

15th November 2008, 18:15

Please read the thread. It has nothing to do with anydvd. it is about opensource BD+ implementation.
Later

Edit:
James for anydvd says that they need at least one month. This is for Blu-ray not dvds. :)

It is the BD that I ripped.

Sorry I thought that doom9 was mentioning Slysoft's inability to break the BD+ on those disks as a method of demonstrating that there is/will be difficulty for the open source project with these lines:
so if you're looking at validating the software against more advanced titles this should provide useful:

October 31st 6.4.7.9 (die another day, the happening, possibly: firefly/bender's game) - Slysoft now says they need 1-2 months to take care of those

dirio49

15th November 2008, 18:41

It is the BD that I ripped.

Sorry I thought that doom9 was mentioning Slysoft's inability to break the BD+ on those disks as a method of demonstrating that there is/will be difficulty for the open source project with these lines:

True, i think maybe there was another BD.
well here is the list
http://forum.slysoft.com/showthread.php?t=21985

BTW did you try to rip the BD with the tool found here?

loo3aem3ON

15th November 2008, 20:16

A new obfuscated aes implementation is being used which looks a bit more complex than the previous one. But i guess it will be no problem for you to break again :rolleyes:

Should I upload the contents of BDSVM somewhere? (excluding the BACKUP folder).
Yes please. Furthermore i need to you rip the disc with anydvd and dumphd (without the bd+ option) and tell me the first few differences you see. For example the first four 5-byte blocks which are different including their address. This is for me to verify that the repair descriptors of the first nonempty segment are correct. You can use WinHEX or similar to automatically compare the files.

thank you

Edit2: I have read this thread thoroughly, but i'm unaware as to what flash.bin is used for, how it is obtained/created and where it's needed, if needed. I haven't needed it yet and BD+ is removed without any problems.
It's the slot memory. If the content code wants to store data in a nonvolatile memory region of the player it uses the slot memory. This can for example be used to collect data about your viewing habits or to restrict how many times you may watch a particular movie.

saint-francis

15th November 2008, 21:19

BTW did you try to rip the BD with the tool found here?

No I didn't. I haven't tried it yet.

BTW the BD I ripped was the happening. It is on doom9's list but not on the one you linked to.

xkodi

15th November 2008, 21:27

Ice Age (US version) BDSVM folder (without the BACKUP folder):

http://rapidshare.de/files/40917565/BDSVM_ICE_AGE_US.rar.html

first four different 5-byte blocks between BD+ aware and BD+ unaware rip of 00001.m2ts and their addresses:

1. file offset 0x00045B9D:

DA D6 C6 16 1D (AnyDVD rip, i.e. BD+ removed)

99 50 D6 AD A4

2. file offset 0x00046FE1:

D6 09 61 41 C6 (AnyDVD rip, i.e. BD+ removed)

C3 24 9C B0 05

3. file offset 0x0004B772:

2F A2 81 92 2D (AnyDVD rip, i.e. BD+ removed)

38 07 6C BE 2D

4. file offset 0x0004C858:

FD D5 2D 79 60 (AnyDVD rip, i.e. BD+ removed)

A5 F7 60 9A 9D

loo3aem3ON

15th November 2008, 22:27

3. file offset 0x0004B772:

2F A2 81 92 2D (AnyDVD rip, i.e. BD+ removed)

38 07 6C BE 2D
You probably mean 0x0004B771. Apart from that i can't see any problems. Both repair descriptors which contain the four patches you told me are correct. You can verify this by loading the conversion table in ConvTableView 0.2 (http://rapidshare.com/files/161925290/convtableview-0.2.zip):
1. click on "0x00000002: TableID 00001"
2. click on "0x00000394: Segment 0"
3. look at the table at the right hand side. Address0/1 is the address for the patch "patch0" and "patch1". You may ignore all columns but the last four.

I suggest you check a few more differences yourself. Maybe the problems start in one of a later segments. If you can't find any differences between these conversion table entries and your file comparison you should apply the conversion table to the m2ts file created by dumphd and then compare the result with the m2ts file created by anydvd. Please tell me the first few differences you find.

thank you

xkodi

15th November 2008, 23:03

You probably mean 0x0004B771.

yes, you're right, it's 0x0004B771, not 0x0004B772.

Apart from that i can't see any problems. Both repair descriptors which contain the four patches you told me are correct. You can verify this by loading the conversion table in ConvTableView 0.2 (http://rapidshare.com/files/161925290/convtableview-0.2.zip):
1. click on "0x00000002: TableID 00001"
2. click on "0x00000394: Segment 0"
3. look at the table at the right hand side. Address0/1 is the address for the patch "patch0" and "patch1". You may ignore all columns but the last four.

now i know how to interpret the conversion table. thank you!

I suggest you check a few more differences yourself. Maybe the problems start in one of a later segments. If you can't find any differences between these conversion table entries and your file comparison you should apply the conversion table to the m2ts file created by dumphd and then compare the result with the m2ts file created by anydvd. Please tell me the first few differences you find.

will do all these things and will report the results.

xkodi

16th November 2008, 00:55

will do all these things and will report the results.

finally, full success - no difference with the AnyDVD rip, the two files are byte by byte identical now.

btw, i found out why last time i tried:

http://forum.doom9.org/showthread.php?p=1212526#post1212526

it didn't work - that is because i compiled the content_repair.c with 32 bit fseek function that doesn't support large files. after recompiled content_repair.c with 64 bit fseek it also started to work and produced the same file as dumphd and AnyDVD.

so, BDVM Debugger generates good conv table for Ice Age (US version) and all my troubles were caused by bad compilation of content_repair.c

Accident

16th November 2008, 01:01

Thanks for the files, I do not have the correct volume-id so I don't know if it is going to be correct, but the first couple previously mentioned is correct:

# test/convtab -v 65 -d iceage/ -I 1d2fc67462fdf986357a8f808fa1298b -f dat/flash.bin -t 1 -u 00001.m2ts

segment: direct patch title 0 started.
[segment] index 059F
[segment] flags 42
[segment] adjust0 0030
[segment] adjust1 001B
[segment] offset0 5D
[segment] offset1 61
[segment] patch0 DAD6C6161D
[segment] patch1 D6096141C6
[segment] would seek to 0000000000045b9d to write patch0
[segment] would seek to 0000000000046fe1 to write patch1

Run against a 0-byte files, we get these patches:

00045b90 00 00 00 00 00 00 00 00 00 00 00 00 00 da d6 c6
00045ba0 16 1d 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00046fe0 00 d6 09 61 41 c6 00 00 00 00 00 00 00 00 00 00

0004b770 00 8f 2f a2 81 92 00 00 00 00 00 00 00 00 00 00

0004c850 00 00 00 00 00 00 00 00 fd d5 2d 79 60 00 00 00

0006dd50 00 00 00 00 00 00 00 00 00 00 00 00 79 18 45 8a
0006dd60 86 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0006e820 00 00 00 00 00 00 00 00 00 00 00 3e 52 37 04 8e

000fef30 00 00 00 00 00 00 00 00 00 00 c7 38 1a 9e 15 00

00106190 00 00 00 00 00 00 00 00 00 00 00 00 33 a9 6e 2f

001061a0 60 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00184a30 00 00 00 00 00 35 94 69 89 50 00 00 00 00 00 00

Edit:
I see your edit. excellent.

xkodi

16th November 2008, 01:19

I do not have the correct volume-id so I don't know if it is going to be correct,

at least with Ice Age and Die Hard 4 volume-id is not important for BD+, both with libbluray and BDVM Debugger i use wrong values and conv table is always correct, but maybe this isn't true for other BD+ protected titles.

but the first couple previously mentioned is correct

they are correct, here is my log and the values are the same as yours:

0x00045B9D:
DA D6 C6 16 1D

0x00046FE1:
D6 09 61 41 C6

0x0004B771:
8F 2F A2 81 92

0x0004C858:
FD D5 2D 79 60

0x0006DD5C:
79 18 45 8A 86

0x0006E82B:
3E 52 37 04 8E

0x000FEF3A:
C7 38 1A 9E 15

0x0010619C:
33 A9 6E 2F 60

0x00184A35:
35 94 69 89 50

Edit:
I see your edit. excellent.

i uploaded BDSVM folder from the US release of Ice Age and that release works fine with libbluray, but there are problems with the Australian release:

http://forum.doom9.org/showthread.php?p=1213379#post1213379

so, after MickJT upload BDSVM folder from the Australian release, then you will be able to look at the problem with libbluray.

loo3aem3ON

16th November 2008, 02:34

so, BDVM Debugger generates good conv table for Ice Age (US version) and all my troubles were caused by bad compilation of content_repair.c
Have you tried repairing the movie with dumphd yet? I believe dumphd v0.5 works fine and you would have saved yourself a lot of work trying it immediately like i suggested in my first reply to your "bug report".
Anyway thank you for finding a bug in this little demonstration repair tool. Could you upload the corrected version please? :)

Accident

16th November 2008, 02:49

libbluray uses autoconf and AC_SYS_LARGEFILE, so it should have no such issues.

fopen and big files depend on OS. Some use fseeko, some use fopen(name, "rbF") (or was that the Solaris 256 descriptor problem).. and so on.

MickJT

16th November 2008, 04:36

I'm uploading to rapidshare now, slowly. I'll also test it on Ice Age 2 and see if convtab works on that... and i'm assuming you'd like the Volume ID?

I'll also test it under Linux.

I'm assuming you want me to rip with AnyDVD fully enabled (AACS and BD+ removal), and then DumpHD with just AACS removal, and compare the differences in the .m2ts file? What software is best to do this in? Also, can I use AnyDVD without BD+ removal to do the same function as DumpHD (without BD+) removal?

Perhaps WinHEX can do it.

Edit: Seems to be that the -d path option no longer works?

C:\Users\<name>\Desktop\win32>convtab.exe -d e: -s conv_tab.bin -f flash.bin
Failed to load volume_id.bin. This will fail.
Failed to load C:\Users\<name>\Desktop\win32/BDSVM/00000.svm.

Edit2: Believe it or not, Ice Age 2 doesn't have BD+ protection.

Edit3: OK, the path option works, but when you supply a wrong path or it can't find it, it shows the wrong path in the "Failed to load" line.

Accident

16th November 2008, 05:18

Yeah, if you use "-d path" it will chdir there. I will fix the error print. So is there still a problem?

MickJT

16th November 2008, 05:24

Well, when I stuck in Ice Age 1, which actually has BD+ protection unlike Ice Age 2, the -d path option works fine.. although it crashes of course. I don't know what it does under Linux as i'm still uploading the BSJVM folder which will be done in mere minutes... and i'm ripping Ice Age 1 to the Hard Drive with AnyDVD HD as you requested.

I still have a question about flash.bin. Is it absolutely required to specify it in the command line? convtab also creates one with the -F command, so i'm assuming a blank one is created if you don't specify one, and -F writes it out to a file, which here it's been 128000 bytes exactly. If -f or -F isn't used, a clean one is still used in memory? Is that right? .. and then what use is loading up a previous one? Sorry for the silly questions :)

Accident

16th November 2008, 05:28

That is correct, if you don't specify one, it starts from a blank one. Which may be strange since slot0 at least seems to be formatted. You can use dat/flash.bin in the tarball specifying -f. Don't bother with -F, it is more for comparing changes for developers.

MickJT

16th November 2008, 05:34

Thanks. I'm still trying to figure out the point of loading up dat/flash.bin if it uses a blank one anyway.

The BDSJVM folder has been uploaded here: http://rapidshare.com/files/164219168/iceage1-aus.rar.html

655E67484F36F5734ED4BDDBA3FBC2CAA938ECEB = Ice Age (AUS) | D | 2008-01-24 | V | ABC2457BA0055C2D3AE9A953FE720E0F

Edit: Segmentation fault under Linux. A lot of warnings when compiling, but they seem normal.

Accident

16th November 2008, 08:17

Ah ok, it crashes as I was not checking of ->conv_tab was NULL. Quite easy to fix so it at least runs cleanly, but unfortunately, we do not get a conv_tab.bin at all.

MickJT

16th November 2008, 08:49

I'm guessing it won't be too hard for you to fix since it does work under the bdvm debugger. It's not anything new.

Accident

16th November 2008, 09:15

It uses trap_MediaSHAFileHash(), in which I had many bugs (testing wrong items before even calling it, and incorrect math (>>10 is not / 0x200) and a few others. Interestingly, it even opens files like: "AACS/MKB_RO.inf".

I have another issue that I am looking at now:

[trap] reading '' at pos 00000000013ffe00

It appears I am given empty names. Or perhaps it is supposed to keep the last file open?

Edit:
Sequence goes:

trap_MediaSHAFileHash(0007E6C8, 15, 512, 0007E6B0, 003DFCC8)
[trap] reading 'BDSVM/00002.svm' at pos 00000000013ffe00
trap: read bytes and SHA_BLOCK
8E 5F D3 81 EF 4E 63 5C 98 63 76 38 5C 2E 83 35 B1 FB E4 42

trap_MediaSHAFileHash(00080E1C, 15, 512, 003DFC10, 003DFC0C)
[trap] reading 'AACS/MKB_RO.inf' at pos 0000000000000000
trap: read bytes and SHA_BLOCK
A1 45 9A 34 84 E7 A4 AB E7 2E 50 1F C6 47 FF 83 1F 9A A7 F9

[trap] trap_Aes(KeyID 00000007)
[dlx] TRAP return: R1 = 80000001

[trap] trap_Aes(KeyID 00000008)
[dlx] TRAP return: R1 = 80000001

[trap] trap_Aes(KeyID 00000005)
[trap] trap_Aes(AES_DECRYPT_PLAYERKEYS): 0x207b20->0x218bd4 (1 key 5)
[dlx] TRAP return: R1 = 00000000

trap_MediaSHAFileHash(0007E6C8, 15, 512, 0007E6B0, 003DF920)
FileName: 00:. 00:. 00:. 00:. 00:. 00:. 00:. 80:. 00:. 00:. 00:. 00:. 00:. 00:. 00:.
[trap] reading '' at pos 00000000013ffe00
[dlx] TRAP return: R1 = 80FFFFFF

loo3aem3ON

16th November 2008, 11:47

Interestingly, it even opens files like: "AACS/MKB_RO.inf".
I suggest to record the results of TRAP_MediaSHAFileHash in a file which a user can attach to a bug report. This would eliminate the need to upload files other than those in the BDSVM/ directory.

I don't know which error code should be returned if the file to be hashed can't be found or if the specified section to be hashed lies outside it's borders. Maybe 0x80000001 or 0x80FFFFFF. Let me know if you want me to run a few tests with my player. This should be easy to figure out.

I still have a question about flash.bin. Is it absolutely required to specify it in the command line? convtab also creates one with the -F command, so i'm assuming a blank one is created if you don't specify one, and -F writes it out to a file, which here it's been 128000 bytes exactly. If -f or -F isn't used, a clean one is still used in memory? Is that right? .. and then what use is loading up a previous one? Sorry for the silly questions :)
The flash.bin is primarily for developers. You may use a blank file of the same size which is most likely the case for a new/unused player. The flash contains 500 slots each 256 byte long hence the size of 128000 bytes.

MickJT

16th November 2008, 12:05

What i'm trying to ask, is in what scenario would you need a non-blank flash.bin file to do BD+ decryption? As Accident says, if you don't specify one, it uses uses a blank one in memory, and doesn't write it out to a file (without -F).

That's fine, so, why have it at all? Why do I see examples of usage with -f flash.bin there when you don't even need it? Why is a flash.bin provided? Why is the one provided larger than 128000 bytes?

loo3aem3ON

16th November 2008, 12:23

That's fine, so, why have it at all? Why do I see examples of usage with -f flash.bin there when you don't even need it? Why is a flash.bin provided? Why is the one provided larger than 128000 bytes?
Like i said it is for developers. We can obtain a series of snapshots from any licensed player and compare them with our emulator. In order to achive accurate results we need to download the slot memory prior recording.

Why is the one provided larger than 128000 bytes?
It was initially believed the slot memory consists of 512 slots but now it seems there are only 500. This is why previous version of this file are a bit bigger. All data above 0x1F3FF is currently ignored but might be used later to store other data like the global write counter.

Accident

16th November 2008, 12:39

Actually, I was dd'ing around to make my flash.bin fit that of the snapshots, and I dd'ed one extra at the end. Since slots do not access anything about 500ed, I didn't bother truncating it. It is nothing, please ignore it.

I am not at all sure why it fails still, I have fixed all obvious bugs. Either my ShaMediaFile is still not producing the right contents, or the lazy code in DiscoveryRAM is messing it up.

loo3aem3ON

16th November 2008, 13:24

I am not at all sure why it fails still, I have fixed all obvious bugs. Either my ShaMediaFile is still not producing the right contents, or the lazy code in DiscoveryRAM is messing it up.
Don't waste your time with this. Instead add support for the diffarchive snapshots. The user will be able to _create_ diffarchive snapshots and TRAP_MediaSHAFileHash recordings with a new version of the debugger. But first i will give KenD00 some time to commit his improvements until i start working on this.

Accident

16th November 2008, 14:38

Comparing our implementations of TRAP_MediaSHAFileHash I found this line to interesting to me:

// get the section length
mem = Debugger.vm.ReadMemory32( pLen, 1 );
length = (int) ( mem[0] >> 2 ); // <====

// allocate a buffer to contain the data from the file
byte[] buffer = new byte[ length ];

There is no description for TRAP_MediaSHAFileHash, but that seems to be that the length is actually in 32bit-sized elements, then converted to size in number of bytes. I did not have this, but adding it makes quite a difference. Now the code path is:

trap_MediaSHAFileHash(0007E6C8, 15, 512, 0007E6B0, 003DFCC8)
[trap] reading 'BDSVM/00002.svm' at pos 00000000013ffe00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

trap_MediaSHAFileHash(00080E1C, 15, 512, 003DFC10, 003DFC0C)
[trap] reading 'AACS/MKB_RO.inf' at pos 0000000000000000
FF FF FF FF 00 3D FF 94 00 07 EB 04 00 3D FF 94 00 01 9E F8
[dlx] TRAP return: R1 = 00000000

trap_MediaSHAFileHash(0003C448, 22, 512, 003DFC04, 003DFC00)
[trap] reading 'BDMV/STREAM/00001.m2ts' at pos 00000000fd2c2800
[dlx] TRAP return: R1 = 80000001

Now I don't have the actual stream, but that is more encouraging than a 0-byte filename. I shall tarball it up so we can try it again. I need to do that for the coredump fixes anyway,

Edit:
libbluray-0.0.4.tar.gz (http://uploaded.to/?id=1uqkux) Updated.

loo3aem3ON

16th November 2008, 18:30

Comparing our implementations of TRAP_MediaSHAFileHash I found this line to interesting to me:

// get the section length
mem = Debugger.vm.ReadMemory32( pLen, 1 );
length = (int) ( mem[0] >> 2 ); // <====

// allocate a buffer to contain the data from the file
byte[] buffer = new byte[ length ];

In posting #359 (https://forum.doom9.org/showpost.php?p=1213151&postcount=359) i told you there is a bug in TRAP_MediaSHAFileHash and it looks like you have just found it. The ">> 2" is wrong! Thank you! :)
There also was a problem with sign extension of the offset because both FileOffsetHigh and FileOffsetHigh were accidentally converted to 32-bit integers in the TRAP_handler(). I have compared the results using the jumper snapshots and they seem to be fine now. One thing i noticed though is that the licensed player omitted the hash of the second 0x200 chunk when the whole block was 0x400 bytes long. The hash of the first 0x200 byte chunk was identical only the second hash for the second chunk is not written back to the vm memory. :confused:

Edit: i am holding back my repository updates to make it easier for KenD00 who wanted to commit some patches too.

Accident

17th November 2008, 01:35

Ok, I see you removed the "len >> 2" section so I have gone back with mine as well. Luckily I get the same code path now, when I change TRAP_MediaSHAFileHash to use an internal "SHA_CTX" instead of over-loading the "dst" memory area. It seems "dst" only has space to hold 20bytes and no more, so the SHA_CTX was clobbering the trailing bytes.

I also changed the trap_sha(SHA_BLOCK) method to do the same, just in case.

I don't suppose I could get 512 bytes from iceage1-aus/STREAM/00001.m2ts at byte offset 0xfd2c2800, or the SHA digest of that area.

dd if=STREAM/00001.m2ts of=shablock.bin bs=512 skip=8295956 count=1

Edit:

Thanks to MickJT for sending me sha_block2.bin (completely untouched block) I now have the following results:

# time test/convtab -d iceage1-aus/ -s saved_conv_tab.bin
real 0m1.194s

[segment] Starting decode of conv_tab.bin: 0x4dfafc (1014544)
[segment] num tables 29
[segment] Table 0 ID 00000001, 227 segments.
Segment 1 offset 00000398 -> 184 entries
Segment 2 offset 000011FC -> 148 entries
Segment 3 offset 00001D90 -> 184 entries

-rw-r--r-- 1 owner group 1014544 Nov 17 16:17 saved_conv_tab.bin

MickJT

17th November 2008, 05:07

I've sent you the files you require, but i'm not sure skipping 8 megabytes is enough. BD+ only seems to start happening after the 20th Century Fox logo.

Rupan

17th November 2008, 05:51

It seems to me that having the volume ID stored in a binary file is not too usable for a large audience. Asking someone to open a hex editor, when the tools that are readily available to recover these values output hex strings, is a major roadblock for the normal user.

I realize that there is a menu option to input the key in the GUI. But I think a better option would be to store the VUK and other encryption parameters in a separate file that is human-editable. This will allow easy updates in case e.g. keys are revoked, so that people do not have to rebuild the software in order to change them.

To these ends I have written a simple FileIO class to add accessibility to the bdvm. I think it would be a good idea to write out a basic configuration file, possibly with headers like

[Known VUKs]
$DiscID $VUK

I'll leave it to you to take or leave this as you see fit.

oibaf

17th November 2008, 11:32

libbluray-0.0.4.tar.gz (http://uploaded.to/?id=1uqkux)

I noticed that the readme.txt file says:
It is released anonymously to protect the author and
without any license.

OK for the anonymity, but releasing it without a license make the library not usable by anyone. It should be released at least under public domain, while with a GPLv3 or later, free projects could use it while avoiding that improvements will be kept secret under a closed source license.

Accident

17th November 2008, 12:23

Pick a license that you think would be best, and I'll put it on the files I made. I've used New-BSD in the past, but I care not about these things.

Heh evvktvh (http://www.evvk.com/evvktvh.html#english)

oibaf

17th November 2008, 13:01

Heh evvktvh (http://www.evvk.com/evvktvh.html#english)
:)

Pick a license that you think would be best, and I'll put it on the files I made. I've used New-BSD in the past, but I care not about these things.

While I can't say nothing since I did not write a line of that code, I would suggest GPLv3 or later, since:

it prevents others taking the code and re-releasing it as closed source software (we already have AnyDVD with its secret, closed source implementation);
it also prevents Tivoization (http://en.wikipedia.org/wiki/Tivoization) (permitted by GPLv2).

loo3aem3ON

17th November 2008, 15:00

Here is a snapshot package of "Jumper" from a player which uses the current publicly available key set.

Jumper v1.01 (http://uploaded.to/?id=al3531) *new*
Jumper v1.00 (http://uploaded.to/?id=9kqx1z)

The content code does a lot more checking which should help us finding some of the hidden bugs.

Also note that trap#8010 is sending this message to the console:
2008-10-17 02:50:19.82 playback starts

Disabled

17th November 2008, 16:40

First I want to give you all a big thumbs up for your work! I'm sorry I just left your project but felt like there were better developers involved now.
I wanted to comment about the licence. I don't think GPLv3 is a suitable licence for this project. I don't think Tivoization will be a problem, because if someone legally sells hardware with this lib he has to comply with the BD licencing authority and I don't think they will allow this lib to be used. And if you use v3, GPLv2 only software won't be able to use this lib. I wouldn't even mind, if someone writes a free but closed source app using this lib (commercial would again be a problem because of the BDLA) so I would choose the "LGPLv2.1 or later" licence, but if you don't want to support closed source software I would use "GLPv2 or later". The "or later" is important that GPLv3 software like dumphd can use it without problems. I tried to search GPLv2 only software but many players (like mplayer) just state its licence as GPL without a version, so it would at least be unclear if it could link to a GPLv3 licenced software.
As far as I can see libaacs is under an evvktvh style licence.
To sum LGPLv2 up as I understand it: You have to release any changes you make to the lib, but your program can have any licence you want.

loo3aem3ON

17th November 2008, 16:47

And another bug found. This time in:
UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);
The content code tells the player how many bytes of memory are reserved for the answer it expects. This length is stored at <*len>. The player checks if the space is sufficient and after writing the answer it updates the length with the number of bytes it has written. The issue is fixed in revision 138.

Edit: simplified and correct the explanation above. :rolleyes:

MickJT

18th November 2008, 02:11

The issue is fixed in revision 138.

Sorry for asking, but of what, and where?

Accident

18th November 2008, 02:14

I would have expected that all traps that take a pointer to uint32_t len, expects it modified in return. For example, TRAP_LoadContentCode, if the read is short. Just a guess though. Why make it a pointer if it isn't expecting a return value.

Edit:
Never mind, turns out I already do. I should change trap_DeviceDiscovery to check there is enough room as well though.

loo3aem3ON

18th November 2008, 03:09

I would have expected that all traps that take a pointer to uint32_t len, expects it modified in return. For example, TRAP_LoadContentCode, if the read is short. Just a guess though. Why make it a pointer if it isn't expecting a return value.
So far we only used this to return the <number of bytes written> but we never read the length from this address.

I should change trap_DeviceDiscovery to check there is enough room as well though.
That's exactly what the debugger was missing too. :)

Next problem:
UINT32 TRAP_MediaSHAFileHash(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst);
*edit* This system call has to check how many chunks it can read from that file position and update the length at <*len> accordingly before returning. */edit*

Edit: corrected description of problems with TRAP_MediaSHAFileHash above. It was originally stated that this system call would only produce one hash (0x14 bytes) and hashing only the first chunk (0x200 bytes). This wrong statement was due to insufficient testing and the lack of any description about this trap.

Accident

18th November 2008, 03:18

So TRAP_MediaSHAFileHash only ever does _one_ hash, or, always return the hash-block size 0x200, but produce all digests requested?

Edit:

Never mind the date thing. "2008" is not two bytes with "20" and "08". Sigh

Also, it is entirely possible the only does ONE hash is because of:

[trap] reading 'BDSVM/00002.svm' at pos 00000000013ffe00
trap: read bytes and SHA_BLOCK
[trap] shaMediaFile warning short read

It was only able to read ONE block 0x200 after all, then it is at EOF.

Edit2:

After those little fixes, Jumper gets further but I do not think I can make it complete, without at least two shablocks of the main file:

[trap] reading 'BDMV/STREAM/00005.m2ts' at pos 00000000ae8d5200
[trap] reading 'BDMV/STREAM/00005.m2ts' at pos 00000002a84dce00

dd if=00005.m2ts of=shablock1.bin bs=512 skip=5719721 count=1
dd if=00005.m2ts of=shablock2.bin bs=512 skip=22292199 count=1

It dies soon after. But I take this as good news.

loo3aem3ON

18th November 2008, 13:35

Also, it is entirely possible the only does ONE hash is because of:

[trap] reading 'BDSVM/00002.svm' at pos 00000000013ffe00
trap: read bytes and SHA_BLOCK
[trap] shaMediaFile warning short read

It was only able to read ONE block 0x200 after all, then it is at EOF.
All the time i thought it was hashing 00002.m2ts which is of course much bigger. Now that was stupid. The debugger also never checks how many bytes it was able to read from that position. :rolleyes:

After those little fixes, Jumper gets further but I do not think I can make it complete, without at least two shablocks of the main file:
Here (http://uploaded.to/?id=i4vb1p) they are. Also if you prefer the original full snapshots you can still extract them from the diffarchives using the diffarchive tool from the repository. You can select this from the menu once you start the tool. :)

Accident

18th November 2008, 13:56

Thanks for the blocks, libbluray-0.0.5.tar.gz (http://uploaded.to/?id=a6nfwc) can handle Jumper as well.

[segment] Starting decode of conv_tab.bin: 0x4dfafc (1035732)
[segment] num tables 31
[segment] Table 0 ID 00000000, 1 segments.
[segment] Table done. Setting ptr to 00000010
[segment] Table 1 ID 00000001, 1 segments.
[segment] Table done. Setting ptr to 0000001E
[segment] Table 2 ID 00000002, 5 segments.
[segment] Table done. Setting ptr to 0000004C
[segment] Table 3 ID 00000003, 8 segments.
[segment] Table done. Setting ptr to 00000092
[segment] Table 4 ID 00000004, 11 segments.
[segment] Table done. Setting ptr to 000000F0
[segment] Table 5 ID 00000005, 446 segments.
Segment 0 offset 000007EE -> 3 entries
Segment 1 offset 0000082E -> 58 entries
Segment 2 offset 00000CBA -> 67 entries

-rw-r--r-- 1 owner group 1035732 Nov 18 21:52 jumper_conv_tab.bin

real 0m1.535s

Edit;

Uploaded

loo3aem3ON

18th November 2008, 14:31

Thanks for the blocks, libbluray-0.0.5.tar.gz (http://uploaded.to/?id=a6nfwc) can handle Jumper as well.
The debugger can't handle "Jumper" currently. The segment keys are wrong but this doesn't mean yours have to be wrong too. Actually the main reason i bought this movie for was to find bugs and not to add quick support for it. You should also work with the snapshots to test your implementation. For example TRAP_Sha is still wrong and TRAP_DiscoveryRAM is giving wrong results too.

Accident

18th November 2008, 14:42

Ah I suppose there is no way to know if the segment keys are wrong without snapshot use? Hmm I guess getting offsets that aren't strictly increasing in order would imply the values are incorrectly descrambled.

KenD00

18th November 2008, 15:04

Q:
Ah I suppose there is no way to know if the segment keys are wrong without snapshot use?

A:

Hmm I guess getting offsets that aren't strictly increasing in order would imply the values are incorrectly descrambled.

And in the way how the patch offsets are calculated i assume that the patches shouldn't exceed TS packet boundaries too. A quick check with ConvTableView tells me that my Jumper conv_tab.bin is indeed not correctly decrypted :(.

:rolleyes:

IceM

18th November 2008, 20:04

Nice project and nice progress! Im a developer myself and since im lazy I was wondering what hardware was needed to contribute to this project? (e.g. hardware needed for testing). Any special drives that require firmware patching or does any bd drive do?
About the licensing, please read the TOS of assembla, if I remember correctly they had a few things in there that stated that they get rights on your code once you commit it to the svn.

loo3aem3ON

18th November 2008, 20:41

Ah I suppose there is no way to know if the segment keys are wrong without snapshot use? Hmm I guess getting offsets that aren't strictly increasing in order would imply the values are incorrectly descrambled.
There are some checks done (e.g. if the first two bits of the flags byte are either '01' or '10') but i haven't studied this yet. I suggest to make a list of problem i should look into (with priorities) and put it into the repository where it gets updated.

Im a developer myself and since im lazy I was wondering what hardware was needed to contribute to this project? (e.g. hardware needed for testing). Any special drives that require firmware patching or does any bd drive do?
We obtain snapshots and traces of the vm state from licensed players which we use for automatic testing and to force our emulators to "stay on track". If an emulator enters a different state than the licensed player we evaluate the differences (console debug output) and "correct" the state before resuming execution.
So with these snapshots/traces you can write your own emulator without any blue ray drive or disc. Later you probably want to test your implementation with content code from a disc we have no snapshot/trace packages for. Then you would need a blue ray drive for which volume id firmware patches are available (see this (https://forum.doom9.org/showthread.php?t=139522) thread),

A sample snapshot/trace package can be found in posting #402 (https://forum.doom9.org/showpost.php?p=1213980&postcount=402).

About the licensing, please read the TOS of assembla, if I remember correctly they had a few things in there that stated that they get rights on your code once you commit it to the svn.
We should put the code under GPL and hope nobody complains. I don't think the license makes a big difference though.

IceM

18th November 2008, 23:25

And with licensed players you mean software players like windvd/powerdvd?

Accident

19th November 2008, 02:06

I don't suppose we could get the first (say, 5?) correct segment keys posted for Jumper? I have been looking at what it does with each Segment key request, and I am quite surprised that each one is quite different. 5/0 Calls lots of Aes(). 5/1 AddWithCarry+Memmove, 5/2 MemSearch, 5/3 SHA, and so on. How tedious for them.

Anyway, during the testing of Aes, there is a whole bunch of tests that should fail, but one succeeds that I suspect should not:

[dlx] Aes(003DEA6C, 003DEA4C, 00000002, 003DEA54, FFF10000

*dst and *key overlap (and even *src, but perhaps that is valid). I have added an additional check with "dst+len vs key" so mine fails.

My segment keys are currently:

27 AF 07 95 76 A7 B6 55 88 16 46 0C 37 85 06 3B
9C 33 19 F9 E9 E5 01 51 5F F1 9F 4C CB 34 19 A4
CE 5D 01 4C 82 D2 2A 45 90 A1 B6 D0 F2 F1 4A 2C
10 DC 7C 22 31 05 8C 63 12 57 28 8B 3B 96 C1 CF
A3 BF 52 C8 7D F3 88 D6 2B 10 7C 19 FD 37 C1 BF

5/0 Segment _could_ be valid, but 5/1 definitely seems wrong (its offset is before 5/0)

js06078

19th November 2008, 02:49

Accident: See posting #213 (http://forum.doom9.org/showthread.php?p=1193182#post1193182) for the behaviour of the segment-key-computing events. Basicly, every event 0x220 is repeating different parts of the previously (i.e. during VM-initialization or after event 0x110(1) was invoked) executed trap-checking before it reads from slot 0 and then computes the segment key (of course only if the trap-testing was 'succesful', i.e. the player behaved like a licensed player).
However, notice that the BD+ developers may change the content code so that these events could be treated completely different in future BD+ versions. But until now, each invocation of event 0x220(<m>,<n>) is testing different traps for increasing values of <n> (and/or <m>, actually).

Another information that I didn't see in this thread yet: Section 1 of the file 00002.svm holds the encrypted conversion table for all the movies that have been looked at in this thread.

Also, here is an analysis of the snapshots for "Jumper" (Debugger Revision #143 was used to create the analysis):

- Trap_DiscoveryRAM is still giving quite a few errors. The first call with errors is probably due to an updated player-firmware on the snapshot-creator's system. Here is one of the other problematic calls:
#004114 TRAP_DiscoveryRAM (00254E23, 003DFCA0, 00000080);
[!] memory difference after trap call: 3DFCA0: 6A (vm) != 5C (snapshot)
[!] memory difference after trap call: 3DFCA1: C7 (vm) != 31 (snapshot)
[!] memory difference after trap call: 3DFCA2: 9E (vm) != 0C (snapshot)
[!] memory difference after trap call: 3DFCA3: EF (vm) != ED (snapshot)
...
[!] memory difference after trap call: 3DFD1C: DE (vm) != 44 (snapshot)
[!] memory difference after trap call: 3DFD1D: CC (vm) != 01 (snapshot)
[!] memory difference after trap call: 3DFD1E: 9B (vm) != C4 (snapshot)
[!] memory difference after trap call: 3DFD1F: 36 (vm) != 8D (snapshot)

- Trap_SHA has a bug (which got even worse with revision #143). This bug seems to appear, when more than 512 bytes are being added to the SHA-context, such as here:
#000502 TRAP_Sha (003DFD8C, 00001180, 00001000, 00000001);
[!] memory difference after trap call: 3DFD8C: 01 (vm) != E1 (snapshot)
[!] memory difference after trap call: 3DFD8D: 23 (vm) != D0 (snapshot)
[!] memory difference after trap call: 3DFD8E: 45 (vm) != 1A (snapshot)
[!] memory difference after trap call: 3DFD8F: 67 (vm) != 82 (snapshot)
...
[!] memory difference after trap call: 3DFD9D: E1 (vm) != 92 (snapshot)
[!] memory difference after trap call: 3DFD9E: D2 (vm) != FC (snapshot)
[!] memory difference after trap call: 3DFD9F: C3 (vm) != 5B (snapshot)
[!] memory difference after trap call: 3DFEE9: 00 (vm) != 80 (snapshot)

- Trap_DeviceDiscovery is (on ~4 occasions) still being executed even though it shouldn't be executed according to snapshots (probably due to bad length-value at <pLen>):
#005735 TRAP_DeviceDiscovery (00000001, 00000001, 003FFD48, 003DF814);
[!] register difference after trap call: r01: 00000000 (vm) != 80000001 (snapshot)
[!] memory difference after trap call: 3DF816: 01 (vm) != 02 (snapshot)
[!] memory difference after trap call: 3DF817: 1C (vm) != BC (snapshot)
[!] memory difference after trap call: 3FFD49: 00 (vm) != 01 (snapshot)
...
[!] memory difference after trap call: 3FFE60: FA (vm) != 00 (snapshot)
[!] memory difference after trap call: 3FFE61: F3 (vm) != 00 (snapshot)
[!] memory difference after trap call: 3FFE62: 72 (vm) != 00 (snapshot)
[!] memory difference after trap call: 3FFE63: 6F (vm) != 00 (snapshot)

- Trap_DeviceAccess is not implemented, but one call of this trap (with 'new parameters') actually writes a result to VM memory now:

#004650 TRAP_DeviceAccess (00000001, 00000000, 003DFB70);
[W] TRAP_DeviceAccess not implemented!
[!] memory difference after trap call: 3DFB70: 00 (vm) != 0F (snapshot)
[!] memory difference after trap call: 3DFB72: 01 (vm) != FF (snapshot)
[!] memory difference after trap call: 3DFB73: 00 (vm) != FF (snapshot)

The same events as before are called for Jumper, with one exception:
- event 0x110(0,0x13)

is used for decryption of the conversion table (?) instead of
- event 0x110(0,1)

This could mean, that the sub-table with id=0x13 needs to be decrypted instead of the one with id=1. This would be reasonable if the sub-table with id=0x13 holds the repair-descriptors for the protected parts of the disc. :confused:

Also notice, that Trap_0x20 (usually signalling the adress and length of the decrypted conversion table to the player) is never being called with other parameters than (0,0x1000) in the snapshot-package for "Jumper" :confused:

loo3aem3ON

19th November 2008, 03:43

I don't suppose we could get the first (say, 5?) correct segment keys posted for Jumper?
Why not? Everything is recorded in the snapshot/trace packages. See posting #402 (https://forum.doom9.org/showpost.php?p=1213980&postcount=402) and download the new Jumper snapshot package v1.01.

The first 5 segment keys are:
62 9E 6A 71 B7 75 62 00 98 9C 7D E6 6A 35 2D 5D ( subtable 5, segment 00)
90 85 E7 C4 92 05 A8 EF 6A AB D3 0C 6A 86 09 29 ( subtable 5, segment 01)
E5 E8 5D 4F FE AC 6B 39 2B 04 04 44 D8 A4 A0 51 ( subtable 5, segment 02)
6D C7 8E 75 6F 32 46 01 5E A6 B4 B7 81 82 12 55 ( subtable 5, segment 03)
E0 D5 F7 C5 6C 62 E8 D6 C4 0E DA 87 4B AF 8A 29 ( subtable 5, segment 04)

Anyway, during the testing of Aes, there is a whole bunch of tests that should fail, but one succeeds that I suspect should not:
Like i said please make a list of things you want me to check. Apart from that please use the snapshots because they are correct.

Also notice, that Trap_0x20 (usually signalling the adress and length of the decrypted conversion table to the player) is never being called with other parameters than (0,0x1000) in the snapshot-package for "Jumper" :confused:
That's because the snapshot/trace package was too short. A more extensive version is avaliable in posting #402 (https://forum.doom9.org/showpost.php?p=1213980&postcount=402).

loo3aem3ON

19th November 2008, 14:53

I've fixed a bug in the parameter checking of TRAP_DeviceDiscovery (thanks js06078) and another bug in TRAP_Sha (thanks [anonymous]). It seems the segment keys for jumper are correct now so i will make this a new release:

Download: BDVM Debugger v0.1.2 (http://uploaded.to/?id=40hx0q)

Please report back and tell us if you experience any problems.

loo3aem3ON

19th November 2008, 16:46

There might be an issue when repairing a movie with the correct conversion table. I've compared the results and some repair descriptors are ignored by AnyDVD-HD. Either these descriptors insert invisible watermarks for the industry to trace any illegal copies back to a player or these descriptors are invalid (maybe a parity bit in the 'flag' byte?) and would only corrupt the movie even more.

This (http://uploaded.to/?id=61qui7) archive contains the conversion table i get for "Jumper" as well as a list of the first few descriptors which were not applied by AnyDVD-HD.

Edit: fixed link

js06078

19th November 2008, 17:39

While taking a look at "Jumper v1.1" snapshots, i noticed, that there are some additional invocations of event 0x110 with different parameters (0x13,2,0x14,3). They all do similar trap-testing but do not create a conversion table. This seems to be obfuscation tactics but I guess it can be easily skipped (by completely ommiting invocation of these events). However, the last event before the playback of the movie starts is event 0x110(1) and it loads the conversion table (Length: 0x000FCDD4, Offset: 0x002DFAFC).

Here are some more bugs that I noticed:

#018343 TRAP_Aes (003DE214, 003DE224, 00000000, 003DE1C4, FFF10000);
[!] register difference after trap call: r01: 00000000 (vm) != 80FFFFFF (snapshot)

This happens once per each call of event 0x220(5,n):
#018346 TRAP-SlotRead (003DF968, 00000000);
[!] memory difference after trap call: 3DF9B2: 01 (vm) != 00 (snapshot)
[!] memory difference after trap call: 3DF9B3: 13 (vm) != 2E (snapshot)

I guess it happens since the slot-treatment still uses data from an older player-version.

One more small bug:
#018619 TRAP_MemSearch (003BFA5C, 00000014, 003BFA6E, 00000002, 003BF99C);
[!] memory difference after trap call: 3BF99D: 00 (vm) != 3B (snapshot)
[!] memory difference after trap call: 3BF99E: 00 (vm) != FA (snapshot)
[!] memory difference after trap call: 3BF99F: 00 (vm) != 6E (snapshot)

None of these bugs seems to be crucial for correct conversion table creation and decryption, but I think it would be best to make the debugger behave as identical to the player as possible.

loo3aem3ON

19th November 2008, 18:27

While taking a look at "Jumper v1.1" snapshots, i noticed, that there are some additional invocations of event 0x110 with different parameters (0x13,2,0x14,3). They all do similar trap-testing but do not create a conversion table. This seems to be obfuscation tactics but I guess it can be easily skipped (by completely ommiting invocation of these events).
Event 0x0110 is issued when the playback of a new file starts.
0x13 = 00019.m2ts = Still frame "Mismatch between disc and player region codes. See disc packaging for details."
0x02 = 00002.m2ts = ? (mplayer crashes)
0x14 = 00020.m2ts = Still frame "Thank you for buying ... "
0x03 = 00003.m2ts = a movie trailer
...

Thank you for reporting more bugs.

js06078

19th November 2008, 19:22

About the redundant / ignored repair descriptors:

I noticed that whenever a descriptor is ignored by AnyDVDHD, there would have to be two blocks of 15 consequent bytes replaced (according to the conversion table). Since one descriptor is ignored by AnyDVDHD, one or sometimes both of these blocks get 'destroyed' by the ignoring.

The first five bytes of both 15-byte-blocks are always identical.

Also, the redundant descriptors only appear at the very end of segments.

The first four bits of the flag-byte of an ignored discriptor are always equal to 8 or 9.

Here's a list of the replacements which are listed in the conversion table. Red bytes are being ignored by AnyDVDHD:

Segment 6, Repair descriptor 121:

0x0FD32E5A 1E8A98D1B1 9E6E15DFF1 0FBBC6301
0x0FD48D5A 1E8A98D1B1 9B5B5C1848 0D2FF1F71
0x0FD55B0A 3D5A9618E8
0x0FD64F5E 324617AFFD

Segment 8, Repair descriptor 121:

0x14EFFF9A B47812BEE4 D1A21040BA 651E2AB21
0x14F055DA B47812BEE4 BD45D8F6C5 FAFA45E0B

Segment 18, Repair descriptor 96:

0x2F50675A D873292C18 6A30131FEA 66E24B4B3
0x2F50BB5A D873292C18 89422BF869 875A2840F

Segment 19, Repair descriptor 129:

0x322C71DA A87F313984 4865272D91 74635ECD9
0x322CF39A A87F313984 29A4811976 7DDB59396

Segment 25, Repair descriptor 111:

0x4186B0DA DC741B3DBC D91F56B81D 98B4830A0
0x4187005A DC741B3DBC 75FC8AE136 CA346216C

Segment 33, Repair descriptor 116:

0x54CF45DA E708FA730C 312E60B2E3 B91D2EF2D
0x54CF949A E708FA730C B17E1AF0C8 0F8710CCF

Segment 34, Repair descriptor 91:

0x5706D5DA 02254DAA8F D4341B50C5 08C36DB56
0x57079E1A 02254DAA8F 52B4CFF223 CB6F7B9CC

Segment 41, Repair descriptor 119:

0x68B5871A 158D698E14 1E9D52D154 8E8332D81
0x68B5FB5A 158D698E14 D117525EC1 6F716690A

Segment 44, Repair descriptor 113:

0x70737E5A 65BA46F8C4 3C496C5F9D BF6BE4233
0x7075231A 65BA46F8C4 F2B4A9DA70 CC04B24AC
0x70767B16 2D9BC7F50A
0x70768DB2 0AEA4EB805

Segment 47, Repair descriptor 121:

0x798EB39A 5DCCE19B5E 1E1A5E13A1 A4EACCFE3
0x798F6A9A 5DCCE19B5E 0701561564 11124E630

loo3aem3ON

20th November 2008, 02:39

The first four bits of the flag-byte of an ignored discriptor are always equal to 8 or 9.
It's more difficult than i previously thought. The first byte of the repair descriptor header (currently called "flag byte") consists of two parts. The most significant two bits seem to define the type of the descriptor. They can either be '01' or '10' (meaning type 1 or 2). The remaining bits are a bit address inside a 64-bit array which acts as a mask for the descriptors of type 2:
FB BD FD FF 7F EC DB FF
Now given a repair descriptor it is checked if it is of type 1 or 2. In case of type 2 it is checked if the bit at the given address (specified by the six least significant bits of the flags byte as explained above) is set. If it is set the descriptor is applied otherwise it is ignored.

Example:
The first descriptor that is not applied is:0x00002B1A: Segment 6:
121 0x00151A2D 0x8E 0x13A 0x000 0x1F 0x24 0x0FD48D5F 0x0FD48D64 0x9B5B5C1848 0xB0D2FF1F71
The first byte of the header is 0x8E which means type 2 and bit address 0x0E. So we have to look at bit (0x0E & 0x07) = 0x06 of byte (0x0E >> 3)=0x01. Byte number 1 is 0xBD and the bit 6 is not set (0xBD & 0x40 = 0) so this descriptor is ignored.

I don't know where the array comes from. Most likely from the content code so it should appear at some time in our memory.

Accident

20th November 2008, 15:12

Just reporting in. The bug fixes in Debugger I had already counted for in libbluray, but I had other bugs. One in DiscoveryRAM (Name repeating, memory over-step), Missed (dst < src) test in PrivateKey (?) and DiscoveryRAM len test of 0x100000.

It is still not quite right, but I am slowly getting there. Also, diffarchive support has been added.

Edit:
Bugger, I could not get the Jumper snapshot do to anything right for the longest time, and it only just occurred to me that I downloaded the new Jumper snapshots, but never extracted. Sigh.

One worry at the moment is this call:

[dlx] PrivateKey(00000000, *00005E58, *0000232C, 00000010, 00000000)
[dlx] TRAP return: R1 = 00000000
[bdtest] comparing with 'post_trap_snapshots/post_trap_reg_000720.bin'
[bdtest] comparing with 'post_trap_snapshots/post_trap_mem_000720.bin'
+00005E50 00 00 00 00 00 00 00 00 20 17 4D E3 86 06 D1 78 |..........M....x|
-00005E50 00 00 00 00 00 00 00 00 39 9A 25 75 15 78 26 7A |........9..u.x.z|
+00005E60 CF AE 0C 0C 4D 8A F8 08 1E F2 F6 9B 4E 1F 42 2A |....M.......N.B.|
-00005E60 AF B6 6B 12 29 90 8F 80 82 E1 A3 79 94 49 18 C7 |..k........y.I..|
+00005E70 B4 A0 EB 40 31 22 D2 D8 83 C9 E6 2E A2 39 0E 37 |....1........9.7|
-00005E70 A3 E9 59 FC 26 F1 9A 5C 17 3A 53 A2 BF D9 A1 1F |..Y.......S.....|
[dlx] compare complete. At least 3 differences.

Is PrivateKey allowed to differ, or generally should it not?

Edit:

Thanks loo3aem3ON. I shall not worry about it. The snapshots get me the correct Segment keys finally, I just need to get it to produce it unguided.

loo3aem3ON

20th November 2008, 15:45

Also, diffarchive support has been added.
Now that's some good news! :)

Happy debugging and please make sure you don't implement anything undocumented just to make it work for this snapshot package (like Schluppo did it sometimes). If you have any questions make a list and send it to me. ;)

In the meantime i've studied some attacks on the RSA scheme with small exponents. Because of the content code running on a virtual machine and therefor being very slow the license administration has chosen a very small public key exponent e = 3 in order to improve the speed. It is well know that this leaks half of the most significant bits of the private key d.

The result is:
d'=
5CB9BF8C681B23E2E923E0EAFD02129E20D0137B1F4E411829A2D76F3E004EE9
EFC2FF78D3870835A4E1CB06EEF61911CA0D5F049F2135937834A5566E7520E0
2701D405BE8A18EBB036BBBE19DDA674662EF16CBBBBC72F2C49A93C4417586F
A1E1220331335DA182CDE42B41BD443527EBEFBB7C206268F526191EB2CE05B9
63506A7C9BD0CBFD568DE0AF77479731B050965B87E574CAAFAAF3A3DEBA1CE5
where d' is the approximation of d satisfying:
| d' - d | <= 2 * square_root( N )
Where N is the public key modulus of the BD+ license administration. Finding the private key of the license administration would be a major step towards a break of BD+.

tteich

20th November 2008, 16:38

Now that's some good news! :)

Happy debugging and please make sure you don't implement anything undocumented just to make it work for this snapshot package (like Schluppo did it sometimes). If you have any questions make a list and send it to me. ;)

In the meantime i've studied some attacks on the RSA scheme with small exponents. Because of the content code running on a virtual machine and therefor being very slow the license administration has chosen a very small public key exponent e = 3 in order to improve the speed. It is well know that this leaks half of the most significant bits of the private key d.

The result is:
d'=
5CB9BF8C681B23E2E923E0EAFD02129E20D0137B1F4E411829A2D76F3E004EE9
EFC2FF78D3870835A4E1CB06EEF61911CA0D5F049F2135937834A5566E7520E0
2701D405BE8A18EBB036BBBE19DDA674662EF16CBBBBC72F2C49A93C4417586F
A1E1220331335DA182CDE42B41BD443527EBEFBB7C206268F526191EB2CE05B9
63506A7C9BD0CBFD568DE0AF77479731B050965B87E574CAAFAAF3A3DEBA1CE5
where d' is the approximation of d satisfying:
| d' - d | <= 2 * square_root( N )
Where N is the public key modulus of the BD+ license administration. Finding the private key of the license administration would be a major step towards a break of BD+.
I'd guess you're aware of these papers:
www.cs.umd.edu/Honors/reports/Everstine/390ResearchPaper.pdf
and
theory.stanford.edu/~dabo/papers/RSA-survey.pdf

loo3aem3ON

20th November 2008, 18:13

I'd guess you're aware of these papers:
www.cs.umd.edu/Honors/reports/Everstine/390ResearchPaper.pdf
and
theory.stanford.edu/~dabo/papers/RSA-survey.pdf
I knew the first one. The second one looks promising too.

About the mask: You can find it at address 0x00000020 after event 0x0220. This is directly behind the section key. Every section has it's own mask like it also has it's own section key. The mask for the first section is "0E 95 0C 80 8F A5 F6 A2". Start counting the bits from the right hand side so "A2" contains bits 0-7 and the last byte "0E" contains the bits "56-63". The section masks for first sections are:
0E 95 0C 80 8F A5 F6 A2
77 7F 79 FC 27 DF 86 FB
9F F7 8A FF 3B EF 73 9B
B3 7F BF D7 CF BF DF E7
D4 EB AE 58 FD DD F2 B5
F9 B1 EF 79 5F D6 F7 CF
FF DB EC 7F FF FD BD FB
...
In my previous example the section mask for section 6 was given in reversed byte order so don't be confused.

tteich

20th November 2008, 22:40

I knew the first one. The second one looks promising too.

...
Out of curiosity: is the same key that is used for decryption of the content (AACS title key) also used for the signature verification in BD+? If yes, this would violate the good practices of cryptography (don't use the same key-pair for 2 different things). If not: does BD+ comprise some sort of key revocation and renewal scheme?

js06078

20th November 2008, 23:42

Suggestion about the redundant descriptors:

When each repair-descriptor is XORed with the current segment key (after each event 0x220 in the debugger), one might also check, whether the current descriptor is redundant (just store the current mask similar to the current segment key and then use the current flag together with the mask to decide, whether the descriptor is redundant). If it is indeed redundant, then
a) replace it with the previous repair descriptor, creating two consequent identical repair descriptors and overwriting the bad one
or
b) replace it with the 'empty repair descriptor' which is supposed to be ignored by all repair-tools which might use conv_tab.bin ('empty descriptor' could be all 0x00 or similar).

This would integrate the treatment of redundant descriptors in the tool which creates the conv_tab (debugger) and would allow to keep the repair-tools as simple as possible.

loo3aem3ON

21st November 2008, 03:27

Out of curiosity: is the same key that is used for decryption of the content (AACS title key) also used for the signature verification in BD+? If yes, this would violate the good practices of cryptography (don't use the same key-pair for 2 different things). If not: does BD+ comprise some sort of key revocation and renewal scheme?
The virtual machine and it's interface with the player is defined by the BD+ specification. The conversion table is probably covered too. All the keys we currently have can theoretically be revoked but this could cause backward compatibility problems. What i know for sure is that the new firmware of my player contains a new version key (aes key 6) and a new secondary certificate (with a new ECDSA public key).
If we find a way to forge the signature for the certificates then this would have a significant impact on the security of BD+. We would be able to generate our own ECDSA public/private keys and even change the identity of the player we emulate at will. We already have half of the bits of the private key used to sign the certificates but i'm not sure if this is helping us in any way.

Finally one shouldn't speak of "security" as BD+ is not designed to prevent any attacks. Instead it is supposed to slow an attack down to ensure at least 2 weeks pass before the first pirated copies are sold on the streets. This is the time where most of the money is made by selling authentic copies. At least that's what i've read. :rolleyes:

Suggestion about the redundant descriptors:
Thanks but i've already rewritten some code so the conversion table is now rebuilt after decryption without the redundant repair descriptors.

Download: BDVM Debugger v0.1.3 (http://uploaded.to/?id=j4gz5m)

The movie "Jumper" is now correctly handled

Is PrivateKey allowed to differ, or generally should it not?
ECDSA is using a random parameter 'k' so it naturally creates different (r,s) pairs all the time. So yes TRAP_PrivateKey may differ just like TRAP_Random.

loo3aem3ON

22nd November 2008, 14:17

If not: does BD+ comprise some sort of key revocation and renewal scheme?
A simple revocation scheme could be:
1. call TRAP_DeviceDiscovery to get the player certificates
2. verify the certificates using the LA public key stored in the content code
3. read the player id and version number from the certificates
4. check if the version number is above a certain value. If it is continue decrypting content code specific to this player otherwise halt (call TRAP_Finished and refuse to generate conversion table).

According to the documentation BD+ can employ watermarking techniques so the content code would create a slightly different conversion table for every player. Now it doesn't take much to add up why SlySoft suddenly needs 1-2 months to support new movies. To provide evidence and reveal the marking scheme we need somebody with a different player to present his conversion table. Or i would have to compromise another player.

I see no reason to believe that AACS and BD+ share keys.

loo3aem3ON

22nd November 2008, 22:12

I thought about the attack on the LA RSA private key. So far i haven't found any proof about the security of RSA schemes with low public exponents but there don't exist any publicly known attacks which make use of a low public exponent. If there exists an efficient algorithm which can find the remaining bits of the private exponent (d) then one could efficiently factor N = p * q which is believed to be hard in general. The only additional information we have compared to the more general problem of prime factorization is that the public exponent e (which an attacker knows) and the product phi(N) = (p-1)(q-1) are coprime (RSA requirement).

Currently 1024-bit numbers (products of two about equal sized primes) can be factored in reasonable time using expensive hardware. A 1280-bit modulus (like ours) is out of reach.

haggi

22nd November 2008, 23:59

What about http://boinc.berkeley.edu/? Couldn't we just create a software like that to break that key?

Just getting started with crypto stuff, are there some more documents you can recommend? I would really like to try to help you with this!

Some questions on the key(s) used: how many bits does 'd' have and how many are known by the low exponent attack?

Edit: Maybe we should try debian weak keys first. You never know ...

loo3aem3ON

23rd November 2008, 00:39

What about http://boinc.berkeley.edu/? Couldn't we just create a software like that to break that key?
No way without specialized hardware (asics).

Just getting started with crypto stuff, are there some more documents you can recommend? I would really like to try to help you with this!
That's a good one: link (http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=A33BA4C70978CA934D5726DD122B5CE1?doi=10.1.1.30.8488&rep=rep1&type=pdf). Have a look at fact 3.2 in particular. Btw the introduction in "390ResearchPaper.pdf" (refered to by tteich) is wrong. One n/4 of the least significant bits of d are necessary (not of N).

Some questions on the key(s) used: how many bits does 'd' have and how many are known by the low exponent attack?
About 1280 bits and half of them (the most significant ones) are known.

Edit: Maybe we should try debian weak keys first. You never know ...
I believe it's hopeless and i've tried to explain why in my previous posting. The problem is still hard even if the attacker knows half of the most significant key bits. Remarkably knowing only one quarter of the least significant bits would be sufficient to efficiently compute d and thus factor N. An attacker could obtain these bits from timing or power analysis of the device which creates the signatures.

haggi

23rd November 2008, 02:09

I believe it's hopeless and i've tried to explain why in my previous posting. The problem is still hard even if the attacker knows half of the most significant key bits. Remarkably knowing only one quarter of the least significant bits would be sufficient to efficiently compute d and thus factor N. An attacker could obtain these bits from timing or power analysis of the device which creates the signatures.

I don't think it's entirely impossible. There should be some way to use the most significant 640 bits of the key to efficiently factorize n ... I'm working on something, but i need to talk to a pal - who studied math in university - to verify my equations. They seem to be a bit too simple to be true.

qubic

23rd November 2008, 09:19

No way without specialized hardware (asics).

What about Boinc on GPUs? Isn't that possible?

Accident

23rd November 2008, 09:25

Aahhhh 4 days debugging and it came down to:

result =
trap_Sha(
&vm->addr[ dst & ADDR_MASK4 ], // dst
&vm->addr[ src & ADDR_MASK4 ], // src

Edit:

Still giving wrong results, but I put in a hack to only copy over the bytes (from snapshots) after trap_DiscoveryRAM() and it works, so I just need to fix those memory areas and it is done.

Edit 2:
Current version checked in now produces the correct segment_keys (first 5 at least) but I have not yet dealt with skipping the dummy entries. There are situations that I find strange still, but appears not to matter. There are multiple calls to DiscoveryRAM with src address of 0x0000000. This (should) return the PlayerName. Snapshots confirm this for the first call, but then for following calls, completely different bytes. This is most curious. My code always return the PlayerName.

haggi

23rd November 2008, 11:02

What about Boinc on GPUs? Isn't that possible?

If we don't find some very optimized attack on the remaining key bits, even gpu power won't help much...

loo3aem3ON

23rd November 2008, 20:51

I've restructured our documentation (http://uploaded.to/?id=5442g1) so please have a look at the table of contents. We need more people working on this if possible. Fixing grammar/bad English expressions and typos will help too. Has anyone written a description of the conversion table yet? Would you suggest to move the detailed trap descriptions to the appendix?

loo3aem3ON

24th November 2008, 14:34

I saw you committed a few updates including a patch to add diffarchive support. I would like to comment on this.
// Locate the correct diff. This is tedious. Why couldn't they have
// made the diffarchive with a index-list of offsets.
A diffarchive snapshot is a stream of patches in which you are not supposed to seek. The diffarchive file is opened only once during your program initialization (startup). Then you are supposed to allocate another 4MB of memory and fill it with zeros. Now you have two 4MB memories. One is updated by the virtual machine and the other (lets call it reference memory) is updated by the patches from the diffarchive stream.
Once this is done you can start the virtual machine and let it run until the first trap. In the meantime (different thread maybe?) you can apply the first chunk of patches to the (zeroed) 4MB reference memory which gives you the memory state after the first trap execution.
Now when the first trap has been executed you compare both 4MB memories and if they should differ you overwrite the virtual machine memory with the reference memory.
After this is done you apply the second chunk of patches to update the reference memory to the state after the second trap execution... and so on.

I believe you made it far to complicated. Have a look at the snapshots.java file and see how simple this actually is if you can afford the extra 4MB of memory. ;)

(the register snapshots are threated the same way)

// Seriosuly who writes files in LE, you can't use any of ntohs() etc calls
If you would like to have big endian/little endian swapped in future diffarchive snapshot packages you can try to convince me to change it. :)

Accident

25th November 2008, 00:09

A diffarchive snapshot is a stream of patches in which you are not supposed to seek. The diffarchive file is opened only once during your program initialization (startup). Then you are supposed to allocate

I meant to delete this rant, I wasn't thinking clearly or drunk, one of those two. Of course the differences has to play from the start to get a specific snapshot.

If you would like to have big endian/little endian swapped in future diffarchive snapshot packages you can try to convince me to change it. :)

This one I feel I have some legs to stand on. Since both the OS (since all network is BE), and BD+ itself is BE, we already have functions and support to handle it. But since normal use will not use snapshots, it doesn't matter too much if you can't use them on BE architectures.

But either way, disregard my rantings, they weren't supposed to be checked in, I did so a little bit quickly to have it committed before the end of the weekend. :)

Also, I read your documentation, nicely done.

Edit:

I re-did the cleaning of fake repair descriptors to be done at segment-decrypt time, so they are not in the way for all the functions using conv_tab. I will work on Win32 build today, and all issues that brings. Otherwise, libbluray-0.0.6 should be working with the known titles.

libbluray-0.0.6.tar.gz (http://uploaded.to/?id=3rst84)
If you intend to try it against Jumper, don't forget you need the two shablocks, or the original encrypted 00005.m2ts file.

haggi

25th November 2008, 12:42

My math guy doesn't reply ... So I'll post it here. Prove me wrong if you like :devil:

The signature for some data is calculated as follows:
m = c^d mod n
which is the same as
m = c^d - k*n
with k big enough to satisfy m < n where k is some big int

we know m (the signature), n = p*q (where p and q are the primes we want to get in the end) and c (which is obviously needed the check a signature)

but we want to know d
m + k*n = c^d
d = log_c (m + k*n)
the most efficiently calculated log is the log_2 (the int part is the index of the most significant non-zero bit, counting from the LSB side)
d = (log_2 (m + k*n))/(log_2(c))
where log_2(c) has to be calculated only once

some k has to be found to satisfy
| d' - ((log_2 (m + k*n))/(log_2(c))) | <= 2 * square_root( N )

The last operation (the division by log_2(c)) is to be aborted if the some byte returned is different from the known byte in d'. Which is efficiently possible because a division is calculated from the MSB to the LSB and we know the 640(?) most significant bits. For most k it should be sufficient to only use the int part of both the log_2.

I'm looking into some math libs to see if we could make the whole calculation streamable to save even more computation time if k is wrong, which it will be most of the time :rolleyes:

I will add some pseudo code in the evening... Got to go to work now.

loo3aem3ON

25th November 2008, 14:48

This one I feel I have some legs to stand on. Since both the OS (since all network is BE), and BD+ itself is BE, we already have functions and support to handle it. But since normal use will not use snapshots, it doesn't matter too much if you can't use them on BE architectures.
I'm thinking about it.

But either way, disregard my rantings, they weren't supposed to be checked in, I did so a little bit quickly to have it committed before the end of the weekend. :)
I sensed frustration so i tried to help. Someone should write a proper documentation about how to use the diffarchive files.

Also, I read your documentation, nicely done.
It's far from finished. I'm trying to attract some volunteers to help me.

Otherwise, libbluray-0.0.6 should be working with the known titles.
Did you fix TRAP_Sha to match the snapshots (for all operations)? If yes, i would like to copy your implementation because the intermediates are still wrong here (debugger). :rolleyes:

@all: Please run some tests with the movies you have and report back with the titles of those Movies which don't work. Thank you.

m = c^d - k*n
with k big enough to satisfy m < n where k is some big int
Let m=2 and c=8. Now try to estimate the number of bits 'k' has if you choose the given public modulus N for 'n' and the approximation d' of the private exponent for 'd'.

My math guy doesn't reply ...
He probably crashed due to a memory overflow. :devil:

For most k it should be sufficient to only use the int part of both the log_2.
It's a calculation over a finite field and i don't think you can just use the floor of the logarithm result. Anyway I didn't think about it in detail because k is too large to store anyway.

My suggestion for you is to experiment on a mini rsa example with small numbers and a public exponent of e = 3. Use maple/mathematica/matlab or whatever you have access to.

ala42

25th November 2008, 18:52

[CODE]m + k*n = c^d

First you should estimate the number of digits of c^d and then buy memory, much memory :)

Accident

26th November 2008, 01:17

Did you fix TRAP_Sha to match the snapshots (for all operations)? If yes, i would like to copy your implementation because the intermediates are still wrong here (debugger). :rolleyes:

It is not perfect, but it is better than it was before. I don't know how it compares to Debugger though. Generally byte 349 still differs.
(+addr is libbluray, -addr is snapshots)

[dlx] Sha(003DFD8C, 00002E58, 00002000, 00000002 SHA_FINAL
'post_trap_snapshots/post_trap_mem_000973.bin'
+003DFEE0 00 00 00 00 00 00 00 00 00 00 00 00 15 0D EC C7 |................|
-003DFEE0 00 00 00 00 00 00 00 00 00 80 00 00 15 0D EC C7 |................|

[dlx] Sha(003DEB1C, 003DED1C, 00000001, 00000001 SHA_INIT
'post_trap_snapshots/post_trap_mem_003145.bin'
+003DEB30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
-003DEB30 61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |a...............|

[dlx] Sha(003DEB1C, 003DED1D, 00000002, 00000002 SHA_FINAL
[bdtest] comparing with 'post_trap_snapshots/post_trap_mem_003146.bin'
+003DEB30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
-003DEB30 61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |a...............|
+003DEC70 00 00 00 00 00 00 00 00 00 00 00 00 E1 E1 E1 E1 |................|
-003DEC70 01 00 00 00 00 00 00 00 08 00 00 00 E1 E1 E1 E1 |................|

If one really wanted to, it could be made perfect.

Edit:

I think I have SHA_UPDATE just fine. But SHA_FINAL is harder. It is like they accidentally run one extra turn after final.

+003DEB30 64 63 62 61 68 67 66 65 6C 6B 6A 69 70 6F 6E 00 |dcbahgfelkjipon.|
-003DEB30 64 63 62 61 68 67 66 65 6C 6B 6A 69 6F 6E 6D 00 |dcbahgfelkjionm.|

Edit 2:

Both SHA_UPDATE and SHA_FINAL will process all of Jumper without mismatch. A couple of things are worth noting;
There is sha_ctxt_len set from last update, and then there is the "len" passed to SHA_FINAL. The logic in SHA_FINAL for the reference player leaves "dst" with length from "sha_ctxt_len" only. So "len" is not added on. The bytes left in "dst" are also in "sha_ctxt_len" (without len) in length.

However, this is only true if that length is 44 bytes or less. If longer, it is cleared.

Secondly, OpenSSL's SHA_CTXT after SHA_FINAL, when sha_ctxt_len + len is (a multiple of) "56", is cleared. Which means I do not clear "dst", but leave it untouched.

This makes for some ugly code, and really it isn't required, but it kept me entertained at work. Completely breaks if you change endian.

uint32_t sha_reference(uint8_t *dst, SHA_CTX *sha, uint32_t len,
uint32_t total_len)
{
uint32_t xlen, i;

memcpy(&dst[ 0 ], &sha->h0, sizeof(sha->h0) );
memcpy(&dst[ 4 ], &sha->h1, sizeof(sha->h1) );
memcpy(&dst[ 8 ], &sha->h2, sizeof(sha->h2) );
memcpy(&dst[ 12 ], &sha->h3, sizeof(sha->h3) );
memcpy(&dst[ 16 ], &sha->h4, sizeof(sha->h4) );

// Move all 4 bytes first:
debugf("reference: copying all even 4s from %u\n", len);
i = 0;
xlen = len;
while(xlen >= 4) {
xlen -= 4;
// Refence is big-endian.
memcpy(&dst[ 20+i ], &sha->data[i/4], sizeof(sha->data[0]));
i += 4;
}

debugf("reference: dealing with half-words: %u\n", len-i);

// Deal with the remainder.
switch(len - i) {
case 0:
break;
case 3:
dst[20 + i ] = (sha->data[i/4] & 0xFF00) >> 8;
dst[20 + i + 1] = (sha->data[i/4] & 0xFF0000) >> 16;
dst[20 + i + 2] = (sha->data[i/4] & 0xFF000000) >> 24;
break;
case 2:
dst[20 + i ] = (sha->data[i/4] & 0xFF0000) >> 16;
dst[20 + i + 1] = (sha->data[i/4] & 0xFF000000) >> 24;
break;
case 1:
dst[20 + i] = (sha->data[i/4] & 0xFF000000) >> 24;
break;
}

// Update len field, if needed
if (total_len) {
debugf("reference: updating total size %u\n", total_len);

dst[340] = (uint8_t) ( total_len & 0xFF );
dst[348] = (uint8_t) (( total_len * 8 ) & 0xFF );
dst[349] = (uint8_t) ((( total_len * 8 ) & 0xFF00 ) >> 8 );
}

return 0;
}

..skip skip...

case SHA_UPDATE:
sha_ctxt_len += len;
SHA1_Update(&osha, src, len);
sha_reference(dst, &osha, osha.num, sha_ctxt_len);
break;

case SHA_FINAL:
trap_Sha(dst, src, len, SHA_UPDATE); // Call above UPDATE to add in "len" bytes.
SHA1_Final(digest, &osha);
i = sha_ctxt_len - len; // Compute size before UPDATE added "len".

if (!(sha_ctxt_len%56)) {
memset(&dst[20+i], 0, 352-20-i); // Clear after "i".
sha_reference(dst, &osha, 0, sha_ctxt_len - len);
} else if (i > 44) {
memset(&dst[0], 0, 352); // Clear all
sha_reference(dst, &osha, 0, sha_ctxt_len - len);
i = 0;
} else {
memset(&dst[0], 0, 352); // Clear all, rebuild all.
sha_reference(dst, &osha, i, sha_ctxt_len - len);
}
// Copy over the digest
memcpy(dst, digest, sizeof(digest));
sha_ctxt_len = 0;
break;

It can most likely be further cleaned up if checked against mod 56.

loo3aem3ON

26th November 2008, 20:58

It is not perfect, but it is better than it was before.
Thanks.

Prove me wrong if you like :devil:
We will forgive you if you can't tell us the factorization of N or the private exponent. :)
I've studied a few papers and it is still unknown if factoring is more difficult than the rsa problem. There is some evidence that at least for small public exponents like e=3 it might be easier to solve m^(1/3) mod N than factoring N (or equivalently calculating the private exponent d). But currently nobody knows how to do that without the trapdoor 'd'.

A different approach to forge the certificates returned by TRAP_DeviceDiscovery would be to intervene with the subsequent SHA-1 hash calculation or the encryption of the signature part of the certificate. The interesting data paths (data dependencies) for a given content code program can be found among all automatically recorded data dependencies from the certificate.

xkodi

26th November 2008, 21:24

what about this paper:

http://crypto.stanford.edu/~dabo/papers/ssl-timing.pdf

it describes timing attack on RSA:

Our attack exposes the factorization of the RSA modulus. Let N = pq with q < p. We build approximations to q that get progressively closer as the attack proceeds. We call these approximations guesses. We refine our guess by learning bits of q one at a time, from most significant to least. Thus, our attack can be viewed as a binary search for q. After recovering the half-most significant bits of q, we can use Coppersmith’s algorithm [3] to retrieve the complete factorization.

loo3aem3ON

26th November 2008, 21:52

it describes timing attack on RSA:
I don't have access to any device that contains the private exponent. If i had one i wouldn't be sitting here writing postings ;)
Those certificates are probably created on a pc without a network connection to rule out any side channel attacks.

Schwa226

27th November 2008, 09:06

About RSA attack I can give you following information:

A 512 Bit RSA done with GGNFS:

with GGNFS on 2 A64 X2 4800+ in less then 2 month...
latsieve 11.10. to 14.11.
matbuild/matprune 14.11. to 16.11.
matsolve 16.11. to 30.11.
sqrt 30.11. to 01.12.

Boost GGNFS for Dualcore:

GGNFS adjusted for Intel Core2Duo :Rapidshare (http://rapidshare.com/files/163021086/ggnfs-svn-322-p4-win.zip)

Needed libgmp-3.dll (http://www.dlldll.com/libgmp-3.dll_download.html) DLL

May it helps...

tteich

27th November 2008, 20:29

About RSA attack I can give you following information:

A 512 Bit RSA done with GGNFS:

with GGNFS on 2 A64 X2 4800+ in less then 2 month...
latsieve 11.10. to 14.11.
matbuild/matprune 14.11. to 16.11.
matsolve 16.11. to 30.11.
sqrt 30.11. to 01.12.

Boost GGNFS for Dualcore:

GGNFS adjusted for Intel Core2Duo :Rapidshare (http://rapidshare.com/files/163021086/ggnfs-svn-322-p4-win.zip)

Needed libgmp-3.dll (http://www.dlldll.com/libgmp-3.dll_download.html) DLL

May it helps...

While 512bit brute-force attacks can be done these days, this approach is really useless at a modulus size of 2048bits.

js06078

27th November 2008, 22:46

How about starting work on the newest batch of BD+ titles?

Here's a list of titles that confirmedly employ the newest version of BD+ (which AnyDVD-HD cannot handle yet):

Futurama: Bender's Game (USA)
Firefly, The Complete Series (USA)
Planet of the Apes (1968) (USA)
Planet of the Apes (all of the series) (USA)
Space Chimps (USA)
Meet Dave (USA)
X-Files 2 (USA)
X-Files 1 (USA)
Home Alone (USA)
The Day The Earth Stood Still: Special Edition (USA)

Horton Hears A Who (Hong Kong)

Predator 2 (1990) (UK)
Shine a Light (UK)

loo3aem3ON

27th November 2008, 22:49

While 512bit brute-force attacks can be done these days
The general number field sieve has a subexponential runtime (brute force is exponential). It's the latest and asymptotically fastest algorithm publicly known algorithm for integer factorization. Like all the other previous version (like the quadratic sieve) it is based on fermat factorization. That is finding a pair (x, y) with x>1,y>1 and x!=y which satisfies x^2 = y^2 (mod N). If such a pair is found the factorization of N can be obtained easily (in most cases). Now the gnfs employs very sophisticated techniques to find such pairs which are hard to understand (for me at least). A brute force search is hopeless even for a 160-bit rsa modulus.

this approach is really useless at a modulus size of 2048bits.
Our N (public modulus N = p*q) is 1280 bit long:
N =
8B169F529C28B5D45DB5D1607B831BED31381D38AEF561A43E744326DD00765E
E7A47F353D4A8C507752B08A6671259AAF140E86EEB1D05D344EF801A5AFB150
3A82BE089DCF25618852199D26CC79AE99466A231999AAC6C26E7DDA662304A7
72D1B304C9CD0C724434D640E29BE64FBBE1E7993A30939D6FB925AE0C350896
14F89FBAE9B931FC01D4D10732EB62CA8878E1894BD82F3007806D75CE172B57
Like i said 1024 bit numbers (product of two prims of about the same length) can be factored currently with expensive hardware but not 1280-bit. Maybe with classified algorithms. :rolleyes:

How about starting work on the newest batch of BD+ titles?
People have stopped reporting results so it's probably better to work on the documentation to improve the public understanding of BD+.

Here's a list of titles that confirmedly employ the newest version of BD+
Thanks for the list. Btw. there is only one version of BD+ but each disc can have a unique content code. Every time a content code doesn't run properly on SlySofts emulator they just call it a new version of BD+ but it's not. :rolleyes:

Accident

28th November 2008, 01:12

Anyone able to upload the SVMs of those non-working disks? Top two looks to be most important :)

SixKiller

29th November 2008, 06:05

I`ve got X Files 2 EU laying right here. So what do you need exactly ? I'll upload it.

loo3aem3ON

29th November 2008, 16:44

I`ve got X Files 2 EU laying right here. So what do you need exactly ? I'll upload it.
We need the contents of the BDSVM directory (without the BACKUP subdirectory). If any more data is required we will ask you for it.

Thank you for your support.

kkloster21

29th November 2008, 21:43

i have Firefly the series on blu-ray. how can i send you the BDSVM directories (for each disc)?

SixKiller

29th November 2008, 22:57

Ok so i uploaded this to rapidshare. http://rapidshare.de/files/41026609/xfiles_java.rar.html

Hope you can use this .

Greets

loo3aem3ON

30th November 2008, 00:49

Hope you can use this.
Thank you. I am getting a conversion table which means the key set is accepted. The segment keys are wrong though this is usually due to bugs or more likely in this case the wrong SHA-1 hashes of the files i don't have:
[I] TRAP_MediaSHAFileHash: Hashing AACS/MKB_RO.inf
[I] TRAP_MediaSHAFileHash: Hashing BDMV/STREAM/00008.m2ts

Please try to run the content code in the debugger: DVM Debugger v0.1.3 (http://uploaded.to/?id=j4gz5m).
I am confident that you will get the correct conversion table. You can test it's integrity using ConvTableView which is available from the Dump HD thread.

And it's not java opcode (like your filename suggests you think it is) but DLX like opcode. The content code runs on a DLX like processor. This processor is simulated by our emulator which is implemented in java. Accident has implemented the same emulator in C which runs several times faster.

yippiekayee

30th November 2008, 02:58

I'm not sure if this needs to go into the dumphd thread or this but I've been trying out the bdvmdbg today and ran into some trouble:
I got the conversion table for Prison Break Season 3 Disc 1.. and got no special errors during the process
$ Volume ID set to: 90 BB 43 94 DC C6 50 05 54 FE 96 16 96 07 DE 66
Loading E:\/BDSVM/00000.svm ...
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 6)
[I] TRAP_DebugLog: 2008-11-01 01:43:11.21 playback starts
[I] TRAP_MediaSHAFileHash: Hashing BDSVM/00000.svm
[I] TRAP_MediaSHAFileHash: Hashing AACS/MKB_RO.inf
[I] TRAP_MediaSHAFileHash: Hashing BDMV/STREAM/00004.m2ts
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 7)
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[W] TRAP_DeviceAccess not implemented!
[Event #00000002] 0110 ( 00000000, 00000001 )
[I] TRAP_LoadContentCode: Loading BDSVM/00002.svm (block 0)
Conversion table set

Then I ran DumpHD and it decrypted the file 00001.mt2s just fine as far as I can tell (cursory browsing through the file - by the way, what's the recommended way of quickly checking the decrypted results? can I run md5sums on the Streams directory and compare it with the Streams directory directly on the disc when I have AnyDVD HD running?).
Anyway, DumpHD reports some potential problems
0x0000000000 Decryption enabled
Processing: BDMV\STREAM\00001.m2ts
Error! BD+ SubTable not found, dumping may fail
0x0000000000 Decryption enabled
Processing: BDMV\STREAM\00002.m2ts
Error! BD+ SubTable not found, dumping may fail
0x0000000000 Decryption enabled
Processing: BDMV\STREAM\00004.m2ts
Error! BD+ SubTable not found, dumping may fail
0x0000000000 Decryption enabled

and sure enough the second file is corrupt (this is a series so there's one m2ts file per episode.. there are 4 eps on a disc).. it starts out okay but about 20 seconds in things start to look really bad (reminds me of encrypted DVDs). Since the first file is okay I suspect it has something to do with the episodic nature of the disc.

I loaded the convtable into ConvTableView and it can successfully load the table.

more likely in this case the wrong SHA-1 hashes of the files i don't haveWould it help if he ran sha1sum on those files for comparison? I doubt uploading m2ts files is practical.

yippiekayee

30th November 2008, 03:23

Umm... I think there's a problem with Die Another Day (US).. on the right hand side of the debugger I see some stuff marked in red, though the console output doesn't contain any errors / warnings. The convtable is a lot smaller than my previous ones (592KB instead of almost 1 MB) - convtableview loads the table just fine but there's a lot of emptyness there. I'll post again when the decryption process is through.

loo3aem3ON

30th November 2008, 03:44

I'm not sure if this needs to go into the dumphd thread or this but I've been trying out the bdvmdbg today and ran into some trouble:
It's probably my fault so it's correct to report the problem in this thread.

by the way, what's the recommended way of quickly checking the decrypted results?
I use 'vbindiff' to compare with the result AnyDVD-HD produced. Using md5sum is faster if you only want to check if the files are (still) identical after making changes as developer.

can I run md5sums on the Streams directory and compare it with the Streams directory directly on the disc when I have AnyDVD HD running?).
Yes but you should use a program which shows differences between both files if you have reasons to believe they are different. You can for instance use WinHEX.

Anyway, DumpHD reports some potential problems
Those error messages say that the conversion table doesn't contain entries for all the files you have on disc. It most likely means that either the conversion table is corrupt (which you would have noticed with ConvTableView) or you are using the wrong conversion table. Is it possible that you are using the conversion table from a different movie?

Are you running dumpHD from command line like this example below?
/dumphd-0.5/dumphd.sh --infile:BDMV/STREAM/00001.m2ts --convtable:conv_tab.bin /media/cdrom/ > /tmp/00001.m2ts

and sure enough the second file is corrupt (this is a series so there's one m2ts file per episode.. there are 4 eps on a disc).. it starts out okay but about 20 seconds in things start to look really bad (reminds me of encrypted DVDs).
That's because dumpHD can't repair those m2ts files if it can't find the subtable for the file you would like to restore (AACS decrypt and BD+ repair).

I loaded the convtable into ConvTableView and it can successfully load the table.
Upload the conversion tables please. I would like to take a look at them.

Would it help if he ran sha1sum on those files for comparison? I doubt uploading m2ts files is practical.
There is no need to do that because TRAP_MediaSHAFileHash should be working fine if you have a original encrypted disc. Just make sure AnyDVD-HD is switched off while you create the conversion table.

Umm... I think there's a problem with Die Another Day (US).. on the right hand side of the debugger I see some stuff marked in red, though the console output doesn't contain any errors / warnings.
Those are the registers which have changed. Don't worry about that.

The convtable is a lot smaller than my previous ones (592KB instead of almost 1 MB) - convtableview loads the table just fine but there's a lot of emptyness there. I'll post again when the decryption process is through.
I have recently introduced code which removes bogus repair descriptors. It might be possible that a bug causes the removal of valid descriptors.

yippiekayee

30th November 2008, 04:20

Is it possible that you are using the conversion table from a different movie? No.. it's the right file.. the first of the four episodes is properly decrypted all the way through.. I jumped around the entire 4x minutes and haven't found a glitch.
Are you running dumpHD from command line like this example below?Well I'm on windows but here's a sample commandline (http://forum.doom9.org/showthread.php?p=1218522&posted=1#post1218522). Note that the same procedure has worked for two previous movies (Day after Tomorrow and Hitman).

In the meantime I'm done with Die Another Day and at first glance the decrypted output seems to be fine. DumpHD reported no issues and in jumping around the movie I have yet to discover a corrupted part.

I will compare the decrypted output with the files on the disc when I have AnyDVD HD active.

Are you interested in knowing which discs decrypt okay? I have a sizeable collection (by my count 19 from the list of Fox/MGM discs posted here (http://forum.doom9.org/showthread.php?t=140893) though I'm not convinced every one is really a BD+ disc.. plus two titles are actually series so they have 4/6 discs respectively) and if it helps I can run them all through and compare with AnyDVD HD.

yippiekayee

30th November 2008, 15:33

Yes but you should use a program which shows differences between both files if you have reasons to believe they are different. You can for instance use WinHEX.Could you elaborate on this a bit.. how would I go about doing that and what kind of information would you need to see? I just finished Dr. No and compared sh1sums (I'm back to md5sums now.. it seems to run faster plus it has a progress indicator which is good for impatient people like me).. most files seem to be the same.. there are 6 files (6 KB each) which differ and I'd like to get to the bottom of this.

The same also holds for Live and Let Die.. only that on that disc the difference are 36KB files.

By the way, is it enough to compare the Streams directory or are there other files protected by BD+?

loo3aem3ON

30th November 2008, 16:31

Could you elaborate on this a bit.. how would I go about doing that and what kind of information would you need to see?
Slow down please. I am currently writing a few functions which record the hashes returned by TRAP_MediaSHAFileHash. You will have to run this new version of the debugger once with "Prison Break" and then send me the file "hash_db.bin" it has created.
I believe there is a problem with the callback parameters (event management) and i need the hashes to fool the content code.

Below is the structure of the hash_db.bin in EBNF. Accident will probably want to add support for it. We could include the volume id.
*edit* removed. See posting #480 (https://forum.doom9.org/showpost.php?p=1219113&postcount=480) *edit*

After this issue is fixed we can take a look at the other problems.

I just finished Dr. No and compared sh1sums (I'm back to md5sums now.. it seems to run faster plus it has a progress indicator which is good for impatient people like me).. most files seem to be the same.. there are 6 files (6 KB each) which differ and I'd like to get to the bottom of this.
I would need to see what bytes differ in both files so a hash is useless to me. The conversion table is used to repair the m2ts files. If any other files differ please contact KenD00 who maintains DumpHD.

yippiekayee

30th November 2008, 17:09

I would need to see what bytes differ in both files so a hash is useless to meJust tell me how I can show you that. You suggested Winhex which I installed but I don't really know what to do with it. The files are very small so I suppose I could just upload them for you in this case as well as the convtable but suppose I see a difference in a larger file.. I cannot upload a file that spans multiple GBs.

bugnotme

30th November 2008, 17:33

I don't have access to any device that contains the private exponent. If i had one i wouldn't be sitting here writing postings ;)
Those certificates are probably created on a pc without a network connection to rule out any side channel attacks.

He wasn't highlighting the timing attack but the use of
Coppersmith's algorithm -- a low-exponent attack on RSA --
after the upper half of the bits had been determined by the
timing attack.

Since you claim the upper half bits have been obtained the method
described in the article should be directly applicable. However, I do
not think knowing the upper half bits is necessary in order to apply
Coppersmith's algorithm. I'm also unclear about how knowledge of the
upper half bits can be exploited in order to speed-up the algorithm.

Coppersmith's algorithm is implemented as zncoppersmith in PARI/GP,
coppersmith in Sage and elsewhere.

http://groups.google.com/group/sage-devel/browse_thread/thread/96b4f37e8be967fe/190c9b2a9e8d22ee?hl=en&lnk=gst&q=coppersmith's#190c9b2a9e8d22ee

loo3aem3ON

30th November 2008, 18:33

He wasn't highlighting the timing attack but the use of
Coppersmith's algorithm -- a low-exponent attack on RSA --
after the upper half of the bits had been determined by the
timing attack.
Why would i start a timing attack if i can easily calculate the upper half of the private exponent from the public key (e,N) like i did? I've tried to explain why those bits are most likely useless to factor N. A timing attack can be used to obtain some of the least significant bits. With them N can be factored efficiently.

Just tell me how I can show you that.
Download this snapshot please: http://uploaded.to/?id=7yht2d
Run it once with Prison Break and send me the hash_db.bin and the contents of the BDSVM directory (without the BACKUP subdirectory). That's all i need currently. Before running verify that the hash_db.bin has zero length. Thank you.

haggi

30th November 2008, 19:10

He wasn't highlighting the timing attack but the use of
Coppersmith's algorithm -- a low-exponent attack on RSA --
after the upper half of the bits had been determined by the
timing attack.

But to use this algorithm you need to know the upper half of p or q not the upper half of d, which is what we've got ... :(

yippiekayee

30th November 2008, 21:00

I'm afraid the new version didn't do the trick either. Same errors from dumphd and while the first episode is again properly decrypted, subsequent episodes aren't.

And I did compare the last three of my Bond discs - which played just fine but there are again small files where the md5sums differ. I also watched ripped Hitman today and watched the main movie.. it was glitch free but I haven't yet compared md5sums with AnyDVD.

bugnotme

30th November 2008, 21:06

Why would i start a timing attack if i can easily calculate the upper half of the private exponent from the public key (e,N) like i did? I've tried to explain why those bits are most likely useless to factor N. A timing attack can be used to obtain some of the least significant bits. With them N can be factored efficiently.

Forget timing attacks. The point is Coppersmith's algorithm is a
potentially useful low-exponent RSA attack.

bugnotme

30th November 2008, 21:21

But to use this algorithm you need to know the upper half of p or q not the upper half of d, which is what we've got ... :(

Chapter 6 of this PhD thesis seems pertinent:

http://www.informatik.tu-darmstadt.de/KP/publications/03/bp.ps

yippiekayee

30th November 2008, 21:32

how can i send you the BDSVM directories (for each disc)? Zip them up and upload to a file hoster like rapidshare, uploaded.to, etc and post the link here.

haggi

30th November 2008, 22:05

Chapter 6 of this PhD thesis seems pertinent:

http://www.informatik.tu-darmstadt.de/KP/publications/03/bp.ps

e ∈ [N^(0,5); N^(~0,72))

Our e is 3 which is smaller, much smaller, than N^(0,5)

I am not mathematician enough to know how to alter that equation to suit our needs.

yippiekayee

30th November 2008, 22:59

I just ripped the second disc of Prison Break Season 3. Got the same error about BD+ subtable not being found, but this time only for the episodes... subsequent m2ts files didn't yield the error and the first episode (first m2ts file) was okay again - so I think this is something systemic.. has the BD+ code ever been tested against episodic discs?
This brings me to a question: how can we verify the debugger against titles with MKB versions for which we don't have a processing key yet? Firefly is in the mail but afaik it's MKBv9.. AnyDVD HD can handle that but not the BD+. So, since DumpHD won't be able to decrypt those discs without the processing key is there a way to apply the BD+ removal without doing AACS encryption so that we could first remove BD+, then run AnyDVD HD on it to remove AACS and the verify if the output is correct.

loo3aem3ON

30th November 2008, 23:26

I just ripped the second disc of Prison Break Season 3. Got the same error about BD+ subtable not being found, but this time only for the episodes... subsequent m2ts files didn't yield the error and the first episode (first m2ts file) was okay again - so I think this is something systemic.. has the BD+ code ever been tested against episodic discs?
The problem is the debugger only announces the playback of the first m2ts file to the content code and therefor only gets this portion of the conversion table. It's the largest conversion table i have seen so far which is probably why it was split. I need to rewrite some code to handle this properly.

This brings me to a question: how can we verify the debugger against titles with MKB versions for which we don't have a processing key yet?
AnyDVD-HD will support these titles soon.

is there a way to apply the BD+ removal without doing AACS encryption so that we could first remove BD+, then run AnyDVD HD on it to remove AACS and the verify if the output is correct.
Maybe you have heard of "confusion" and "diffusion" which are properties of every serious encryption algorithm. In our case the "diffusion" property of AES would cripple the entire 128-bit block if you only change a single bit before decryption. I see no way to modify the encrypted stream so that the decryption result is already repaired (without knowing the key of course).

yippiekayee

30th November 2008, 23:55

Oh, I thought BD+ came before AACS, not after. But in this case, if there was a simple BD+ repair program we could fix the stream after decryption by AnyDVD, correct?

loo3aem3ON

1st December 2008, 01:23

Oh, I thought BD+ came before AACS, not after. But in this case, if there was a simple BD+ repair program we could fix the stream after decryption by AnyDVD, correct?
Yes, that is correct.

Try this development snapshot please: http://uploaded.to/?id=ij27kc
I still don't understand the second parameter of callback/event 0x0110 but i know this event occurs before the playback of every m2ts file. So i decided to issue event 0x0110 with the second parameter with all possible values between 0 and 50 to get all the pieces from the conversion table. It seems to work. The resulting conversion table for "Prison break" is 4MB big. ;)

loo3aem3ON

1st December 2008, 19:15

I've redesigned the hash database. It now supports multiple hashes created from a single large block. It's implemented as a chained list which is fast enough for those few entries we have. The structure is:
key = SHA-1 hash of offset, bytesToHash and Filename; 20 bytes
nextPointer = points at the beginning of the next entry; relative address; 4 bytes
bytesHashed = number of bytes used to calculate the hash; 4 bytes

database ::= {entry}
entry ::= key , nextPointer , bytesHashed , {hash}
The reason to create such a database is that the content code uses hashes from arbitrary files on the disc. If a bug prevents the correct creation of the conversion table the developers have problems reproducing the error given only the contents of the BDSVM directory. Because the hash database already contains the hashes the content code likes to verify this is no longer a problem. All the user needs to do is clear the hash_db.bin (e.g. create an empty file with that name) and run the debugger once. The resulting hash_db.bin is then to be sent to a developer.

KenD00

1st December 2008, 19:31

I have also uploaded the files that differ between AnyDVD and DumpHD from Dr. No (US)<link removed>..

Hmm, for some reason AnyDVD zeroed out the end of file 00091.m2ts, but this part looks the same as in the other files, TS packets with mostly FF as content, and the file itself isn't encrypted.

The other files are encrypted and it looks like DumpHD failed to process them. I assume this disc has multiple CPS Units and these files don't belong to CPS Unit 1. In the log of DumpHD should be written something like "Updated Disc data" with a section UM's which has entries which look like <number>[H]-<number>, is there any entry where the second number is not 1? Currently DumpHD can't process discs with multiple CPS Unit keys, maybe because of that your episodic discs don't decrypt properly too.

Those error messages say that the conversion table doesn't contain entries for all the files you have on disc. It most likely means that either the conversion table is corrupt (which you would have noticed with ConvTableView)

But this situation may be perfectly legal too. My observation was that there is a subtable for every file on the disc, but for the files which are not BD+ encrypted these subtables were empty. Maybe its allowed to ommit them in that case?

:rolleyes:

yippiekayee

1st December 2008, 20:09

@Kend00: I didn't keep the logs when I finally shut off my PC last night.. do I have to re-rip everything or will ripping just the files in question do to get the info you need?
Also, any chance for a commandline parameter in DumpHD to do this academic scenario (just fix files with a convtable and not doing any AACS decryption)? That way we could check the debugger against the latest BD+ discs while we wait for a suitable processing key to leak.

yippiekayee

1st December 2008, 20:35

Alright, I started ripping Dr.No again.. here's parts of the output:
Processing disc AACS data...
Volume Unique Key / Protected Area Keys present, decrypting Title Keys / CPS Uni
t Keys...
Searching Title Key / CPS Unit Key Files...
Decrypting e:\AACS\Unit_Key_RO.inf...
Finished decrypting CPS Unit Keys
Updated disc data:
DiscID : 413AF6BB6AF86146B922DCF83D5CF2F6E29EDC8D
Title : Movie Title
Date : 2008-08-14
MEK : N/A
VID / BN's : N/A
VUK / PAK's : 1
0-848FF6607D25510284F0A41ACE852E39
TUK's : 7
1-31D6B69028A16F7544189A1EB7E19D0B
2-85581552DC02679109AF39A4EA5077EB
3-68310D990568522BED70E07295603414
4-FA6FCBC562152AFF69A45FDB064A12B0
5-B409D918375AD199E6984ACE81260BD5
6-BF155D839EEEBFF61609DCF836DFEF6E
7-7F4E414F2EFBB9380507868FA92B14A5
UM's : 1
Set 0
0H-1
1H-1
1-1
2-1
3-1
4-1
5-1
6-1
7-1
8-1
9-1
10-1
11-1
12-1
13-1
14-1
15-1
16-1
17-1
18-1
19-1
20-1
21-1
22-1
23-1
24-1
25-1
26-1
27-1
28-1
29-1
30-1
31-1
32-1
33-1
34-1
35-1
36-1
37-1
38-1
39-1
40-1
41-1
42-1
43-1
44-1
45-1
46-1
47-1
48-1
49-1
50-1
51-1
52-1
53-1
54-1
55-1
56-1
57-1
58-2
59-2
60-2
61-2
62-2
63-2
64-2
65-2
66-3
67-3
68-3
69-3
70-3
71-3
72-3
73-3
74-4
75-4
76-5
77-5
78-6
79-6
80-7
81-7
Sequence Key Block not found (this is good)
AACS data processed

Anything I can do to help to handle those cases? And no, Prison Break only has 1's on the right hand side of the output.

Also, I did test the latest dev build of the debugger and I managed to make a successful rip of the entire disc. I'm now going to rip again using AnyDVD HD and then compare files then proceed to the second disc of the set.

SuperGoof

1st December 2008, 20:39

Currently DumpHD can't process discs with multiple CPS Unit keys

I suggest using Blu-ray Disc Ripper (http://forum.doom9.org/showthread.php?t=129663). It supports multiple CPS Unit keys for about a month now (since v.1.4). I just read the list of CPS Unit keys from Unit_Key_RO.inf, decrypt them, and on the first encrypted block of each .m2ts I try them in turn until I find one which results in correct decryption. (I just check for 16 byte long block of FFFFs within the first 512 bytes of the file :)). And even before doing that, I check if FFFFs are already there, which means that the file is not encrypted at all.

But this is for AACS decryption only, of course. This program does not support BD+ at all yet.

js06078

1st December 2008, 22:42

So, to sum up the results of the recent tests in this thread:

The newest development snapshot of the debugger handles the James Bond movies fine (the remaining problem with the small files on James Bond movies is due to AACS-CPS Unit Keys and not due to BD+), and it also handles Prison Break S03 fine?

This means, that the debugger succeeds even on the newest versions of BD+ content code that also AnyDVD-HD can handle (bringing the open source capabilities up to par to AnyDVD-HD with respect to BD+).

Also, X-Files 2 can not be decrypted by AnyDVD-HD, but the debugger still outputs a conversion table for it. However, this conversion table could still be bad: I took a look at the file, and it contains sub-tables for most of the first 25 .m2ts-files, meaning, that the main feature is split over those files. However, many of the conversion-table segments for these files are empty. Together with the fact, that the non-empty segments do NOT contain descriptors for the 'missing' segments of these files, we can deduce, that either large portions of the files from X-Files 2 are not protected by BD+, or that the created conversion table is not complete and in fact missing segments which are needed for decryption.

So it would be interesting to see, whether it is possible to repair X-Files 2 using the created conversion table. Could someone with the original disc give it a try?

Also I am wondering: If the debugger should really already create a more or less correct conversion table for X-Files 2, why is Slysoft needing so much time to fix their BD+ treatment?

bourke

1st December 2008, 23:57

Also I am wondering: If the debugger should really already create a more or less correct conversion table for X-Files 2, why is Slysoft needing so much time to fix their BD+ treatment?

Perhaps they thought it would be easier just to 'borrow' ideas from this thread - and/or their lead developer is away on annual leave?!

KenD00

2nd December 2008, 00:04

@yippiekayee
You don't need to rip the whole disc or even a whole file, just press the Dump button, then the AACS data gets processed, when the first file gets processed you can abort.

Your log shows indeed multiple CPS Units so this explains the broken files. Currently you can't do anything about it, but you can test the new DumpHD release which will address this, i just don't know when it will be finished.

You can already remove BD+ only, already AACS decrypted files won't get AACS decrypted a second time, but BD+ will get removed. Take your AACS free but BD+ infested rip, copy the AACS folder of the original disc into it, start DumpHD with a conversion table and BD+ will get removed :).

@SuperGoof
I have seen your "brute force" approach to determine the key, i was looking for a better way but unfortunately found no other way than analysing all MovieObjects/Playlists/BDJ-Objects so i will copy your approach :). But i think i won't look for these FFFF's (do you know why they are there and if they always have to be there?), i will check for the TS packet headers. To check if the file is encrypted you can look at the 2 highest order bits of the TP_Extra_Header, if not 00b then the file (Aligned Unit, on BD recordables the encryption state may change inside a file) is encrypted.

:rolleyes:

loo3aem3ON

2nd December 2008, 00:11

This means, that the debugger succeeds even on the newest versions of BD+ content code that also AnyDVD-HD can handle (bringing the open source capabilities up to par to AnyDVD-HD with respect to BD+).
The content code consists of multiple components (startup code, code specific to [player or movie or ( player && movie)], conversion table code...). It's quite a complex piece of software which can fingerprint the player at startup and then select code specific to that player to be executed. So you can't really compare our achievements with those of SlySoft unless the debugger/libblueray and AnyDVD-HD emulate the same player and are therefor executing the same code.

Also, X-Files 2 can not be decrypted by AnyDVD-HD, but the debugger still outputs a conversion table for it. However, this conversion table could still be bad:
The content code will only produce a conversion table if all security checks passed. If you don't see any warnings loading the table in ConvTableView v0.2 then it was almost certainly correctly decrypted. Empty subtables are fine because not all m2ts files are usually corrupted. If subtable x is nonempty then the file x.m2ts was damaged by BD+ and needs to be repaired using that subtable.

Also I am wondering: If the debugger should really already create a more or less correct conversion table for X-Files 2, why is Slysoft needing so much time to fix their BD+ treatment?
Maybe they emulate a different player and have to execute real nasty security checks. The conversion table may also contain repair descriptors which don't do any repairs at all. Instead they insert invisible marks (watermarks) in the decrypted stream which can contain all kinds of information (player id, any data from memory...). The existence of this marking scheme has not been publicly proven yet but i believe it exists. It's are also mentioned in descriptions about spdc which is fairly similar to BD+. So this is something SlySoft might be battling currently. It's probably easily defeated if an attacker has access to multiple different players.

yippiekayee

2nd December 2008, 00:19

So you can't really compare our achievements with those of SlySoft unless the debugger/libblueray and AnyDVD-HD emulate the same player and are therefor executing the same code.So, does that mean that my idea of running the debugger of all my BD+ titles and comparing the output with AnyDVD HD makes sense and I should go ahead (seeing as the process is tedious of it's unnecessary I wouldn't mind - on the other hand, since the process is tedious and may be too complex for people who could potentially help, I'd be happy to contribute something that I can do).

@Kend00: let me know when you're ready.. my discs won't go away and I just ordered the other three Bond discs (the quality of Dr. No really surprised me so I'm game for the old titles). And thanks for the info on processing BD+ without processing AnyDVD.. as soon as Bender's game and Firefly get here I'll run them and report the results.

P.S. The results from disc 2 of prison break look okay, too.

loo3aem3ON

2nd December 2008, 00:37

So, does that mean that my idea of running the debugger of all my BD+ titles and comparing the output with AnyDVD HD makes sense and I should go ahead (seeing as the process is tedious of it's unnecessary I wouldn't mind - on the other hand, since the process is tedious and may be too complex for people who could potentially help, I'd be happy to contribute something that I can do).
Your idea is good and i use the same method to verify the correctness of the conversion table. But to be honest i don't like to see the debugger/libblueray compete with AnyDVD-HD so if you do that in a few months it's fine but not before AnyDVD-HD adds support for these movies. In the meantime you could check if you find a movie for which libblueray is creating a different conversion table than the debugger. Thank you for your support.

yippiekayee

2nd December 2008, 00:57

In the meantime you could check if you find a movie for which libblueray is creating a different conversion table than the debugger.Is there a Windows binary/frontend of libblueray available somewhere?

Accident

2nd December 2008, 01:08

The source tarball contains Windows built binaries for those not setup with MSVC++.
libbluray-0.0.6.tar.gz (http://uploaded.to/?id=3rst84)

To run Jumper, I would run:

win32/# convtab.exe -d jumper/ -i jumper/volume_id.bin -s jumper_conv_tab.bin -v 64
[snip]
[segment] Key 5, 0: 629E6A71B7756200989C7DE66A352D5D mask:0E950C808FA5F6A2
[snip]

-rw-r--r-- 1 owner group 1035732 Dec 2 09:05 jumper_conv_tab.bin

Assuming there is a "jumper/" directory with "jumper/BDSVM/" files etc.

bugnotme

2nd December 2008, 02:49

e ∈ [N^(0,5); N^(~0,72))

Our e is 3 which is smaller, much smaller, than N^(0,5)

I am not mathematician enough to know how to alter that equation to suit our needs.

There are other results in the document which might be useful directly
or point to useful results. Theorem 51-4 on page 99 for instance. That
requires 3/4 of the bits of d, and the difference between p and q to be
big enough.

yippiekayee

2nd December 2008, 20:00

@Accident: is there a documentation of the commandline options somewhere? I cannot get it to work.
I looked at the readme and I think there are the following options:
-d: the source (full path? e.g. e:\ or just the drive letter or what?)
-i: file containing volume id (full path necessary or is it relative to the path from which the application has been started)?
-I: volume id string.. to be used instead of -i
-f: ??
-s: convtable file for the output (full path necessary or relative?)
-v: ??

The program runs and seems to do something but I'm unable to locate the output and there's no return value either.

Here are some commandlines I've tried:
convtab.exe -d e: -I 83569CAE3EEC6ADB278514D894D103B8 -s c:\temp\conv_tab.bin
convtab.exe -d e:\ -I 83569CAE3EEC6ADB278514D894D103B8 -s conv_tab.bin
convtab.exe -d e: -i volume_id.txt -s conv_tab.bin

I've also tried convtab -h, convtab --help convtab -? and replacing - with / in an attempt to get some help but with no results.

update: I finally located the source code and had a look.. sure enough there is a help but it doesn't work.. so something went wrong during compilation. I found that the program seems to do something (use a lot of cpu time at least) if I run it with -d and point it to someplace that has a BDSVM folder, but there's no output whatsoever neither in the console nor in any file.

yippiekayee

2nd December 2008, 23:19

I'm afraid calling Prison Break solved was premature. Since I couldn't run convtab.exe I went ahead with the checksum checking and I have some good news first:

Prison Break Season 3 Discs 1-3 are bit identical with the AnyDVD HD output. The same can be said for
Day after Tomorrow (US)
Hitman (US)
Stargate: Continuum (US)

However, on disc4 of Prison Break Season 3, I get the BD+ subtable not found error again. Here's the link to the BDSVM folder (http://rapidshare.com/files/169650384/PB-s3d4.7z.html) along with the convtable and list of files for which the error occurs. Interestingly though, I compared md5 hashes of all files and the only file where I had a difference in the hash was 00000.m2ts which is the file that contains the single episode on that disc (the rest is extras).. and I cannot play that file at all.

I'm still investigating why from my 3 Bond movies, I can play Die Another Day from the ripped copy despite some undecrypted files whereas I cannot get Dr. No and Live and Let Die to work from the ripped copy (but they work when played from the disc).. I'll check if TMT plays the files from any AnyDVD HD decrypted copy.. if it does, I presume that it's just a matter of using the upcoming DumpHD which supports multiple CPS'es.

Accident

3rd December 2008, 13:37

# convtab.exe -h
options:
-h : display usage help (this output)
-v <int> : enable verbose debug information, 0 to list levels
-d path : specify path to find /path/BDSVM and /path/STREAM/
-s file : specify filename of conv_tab.bin
-i file : specify filename of VolumeID
-I id : directly specify VolumeID
-f file : specify filename of flash.bin
-F : save flash.bin at the end
-t title : specify title of file specified with -u & -U
-u file : directly patch file from conv_tab.
-U : patch stdin to stdout

volume_id.bin (-i), and flash.bin (-f) are loaded first from CWD. Then it chdir() to the -d path. (absolute or relative), All BDSVM, STREAM etc files are opened from -d path POV.
Once finished, it will return to CWD, and write conv_tab.bin (-s) and flash.bin (-fF).

It should be valid to do:

# convtab.exe -i data_files/volume_id.bin -d e:/ -s output/convtab.bin

yippiekayee

3rd December 2008, 19:32

@Accident: that's exactly what I found by browing through the source code but I'm afraid the windows binary you posted does nothing like that.. it just doesn't return any output, be it as a file or via stdout/stderr. Did you actually try out your compiled version? All your commandlines look line Linux commandlines.. so it may be prudent that you try any software for distribution on a native windows system.. While I have compilers in place.. native C++ isn't one of the options I install with Visual Studio so I'm afraid I need a fully tested and working Windows version or I cannot be of any assistance as far as libbluray is concerned.

loo3aem3ON

3rd December 2008, 20:06

@Accident: that's exactly what I found by browing through the source code but I'm afraid the windows binary you posted does nothing like that.. it just doesn't return any output, be it as a file or via stdout/stderr.
Are you running this in cygwin or in the windows command prompt? In the later case please try cygwin.

Regarding your problem with Prison Break Season 4: The conversion table in your archive is from another disc (disc 1?) and you didn't include the hash database. The conversion table created by the content code (BDSVM directory contents you've included) looks fine though. It's probably KenD00's fault that you can't playback the 00000.m2ts :p

dirio49

3rd December 2008, 22:36

nothing to do with cywin
something is wrong with the compiled exe
the previews version run fine,
i just test on the files provided here.

OOPS, I spoke too soon.
it appears to fail for me a least, going by what is available here, because it cannot handle this disk.
it give no console output at all, it just blanks out
on the other alvailabe download that i could find it worked fine

BTW
Accident how do you compile it in linux?
there is not configure files.
i tried autoconfig and automake and no success.
unless I am doing something wrong
thanks

KenD00

4th December 2008, 02:39

It's probably KenD00's fault that you can't playback the 00000.m2ts :p
That can be easily checked by opening the file in a hex editor (but make sure that it doesn't create a backup file when opening :D) and checking if every 0xC0 bytes (well, a couple should be enough) a 0x47 is written, or a bunch of 0xFF and a HDMV at the bigging of the file are OK too :).

i tried autoconfig and automake and no success.
unless I am doing something wrong
thanks
There are Visual Studio Project Files inside win32\libbluray. I have tested the precompiled convtab and it produces a Conversion Table for Die Hard in 1 second and one for Jumper in 20 seconds for me (thats strange, why does Jumper take so long, its almost as slow as the Debugger).

:rolleyes:

dirio49

4th December 2008, 02:59

another test show differences between libbluray 0.06 and bdvm 1.4
but no difference for libbluyray 0.05 and bdvm 1.4
for this files http://rapidshare.com/files/164219168/iceage1-aus.rar.html
uploaded before from somebody else

Accident

4th December 2008, 03:11

BTW
Accident how do you compile it in linux?
there is not configure files.
i tried autoconfig and automake and no success.
unless I am doing something wrong
thanks

In Linux:

# tar -zxvf libbluray-0.0.6.tar.gz
# cd libbluray-0.0.6
# ./configure
# make

Possibly:
# ./configure --with-ssl-dir=/usr/strange/ssl/

OpenSSL needs to be compiled with ECMA, which is standard for the last two years.

dirio49

4th December 2008, 03:21

the repository does not contain a configure file or make file :)

--
thanks that worked, but I was going from the repository

Accident

4th December 2008, 03:51

Ah I didn't know.

Then you would go "autoremake --force --install" and then "./configure". You generally do not check-in the autoconf intermediate files.

bugnotme

6th December 2008, 02:34

Now that's some good news! :)

Happy debugging and please make sure you don't implement anything undocumented just to make it work for this snapshot package (like Schluppo did it sometimes). If you have any questions make a list and send it to me. ;)

In the meantime i've studied some attacks on the RSA scheme with small exponents. Because of the content code running on a virtual machine and therefor being very slow the license administration has chosen a very small public key exponent e = 3 in order to improve the speed. It is well know that this leaks half of the most significant bits of the private key d.

The result is:
d'=
5CB9BF8C681B23E2E923E0EAFD02129E20D0137B1F4E411829A2D76F3E004EE9
EFC2FF78D3870835A4E1CB06EEF61911CA0D5F049F2135937834A5566E7520E0
2701D405BE8A18EBB036BBBE19DDA674662EF16CBBBBC72F2C49A93C4417586F
A1E1220331335DA182CDE42B41BD443527EBEFBB7C206268F526191EB2CE05B9
63506A7C9BD0CBFD568DE0AF77479731B050965B87E574CAAFAAF3A3DEBA1CE5
where d' is the approximation of d satisfying:
| d' - d | <= 2 * square_root( N )
Where N is the public key modulus of the BD+ license administration. Finding the private key of the license administration would be a major step towards a break of BD+.

What did you do to calculate d'?

yippiekayee

6th December 2008, 03:13

I don't know which version of Jumper Oopho2ei tested against - so I took my US copy of Jumper for a quick spin. The 0.1.3 debugger creates a convtable with no problems.. however the 0.1.4 debugger throws an ArrayIndexOutOfBoundsException. The full error is:
java.lang.ArrayIndexOutOfBoundsException: -1621801643
at bdvm.vm.conv_tab_main.setConversionTable(conv_tab_main.java:43)
at bdvm.vm.decoder.getConversionTable(decoder.java:40)
at bdvm.vm.bdsvm_player_interface.TRAP_handler(bdsvm_player_interface.ja
va:658)
at bdvm.vm.BDVM.step(BDVM.java:338)
at bdvm.debugger.Debugger.step(Debugger.java:740)
at bdvm.debugger.Debugger.run(Debugger.java:867)
at bdvm.debugger.Debugger.actionPerformed(Debugger.java:342)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
.... more internal java stuff

The commandline output looks like this:
$ Volume ID set to: 50 4D B5 A0 FC 6F 18 05 47 DF 27 A0 A7 A9 0B E7
Loading E:\/BDSVM/00000.svm ...
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 6)
[I] TRAP_DebugLog: 2008-11-07 02:56:55.37 playback starts
[I] TRAP_MediaSHAFileHash: Hashing BDSVM/00002.svm
[I] TRAP_MediaSHAFileHash: Hashing AACS/MKB_RO.inf
[I] TRAP_MediaSHAFileHash: Hashing BDMV/STREAM/00002.m2ts
[I] TRAP_LoadContentCode: Loading BDSVM/00001.svm (block 7)
[Event #00000000] 0110 ( 00000000, 0000FFFF )
[Event #00000001] 0210 ( 00000000, 00000001 )
[W] TRAP_DeviceAccess not implemented!
[Event #00000002] 0110 ( 00000000, 00000001 )
[I] TRAP_LoadContentCode: Loading BDSVM/00002.svm (block 0)
[I] TRAP_GetConversionTable: Got subtables 2,4,19,20,21,33,34,35,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,
... many events
....
[I] TRAP_LoadContentCode: Loading BDSVM/00002.svm (block 1)
[I] TRAP_GetConversionTable: Got subtables

And some good news.. the entire Die Hard series (US version) decrypted fine and is byte identical with AnyDVD HD.

yippiekayee

6th December 2008, 04:14

it seems convtabd.exe actually works - however, it doesn't seem to like Prison Break Season 3 Disk 4 (and sorry for reporting the problem with that disc.. I indeed picked the wrong convtable)..
I ran convtabd.exe as follows
convtabd.exe -d e: -I 14870BDEF23FD4B9F2E53150B1414041 -s convtab-pbs3d4.bin
And I get an infinite number of
[dlx] unknown trap 00008020:32800.
messages.

Oh, and I'm running this thing natively... cygwin was no good.. didn't offer me access to my BD drive, refused to allow me to create any directory under /cygwin and when selecting another directory mounting would fail.

Accident

6th December 2008, 04:19

That is interesting. It should be just replying with NOT_SUPPORTED for that trap, but I suspect that it has already gone wrong if it ends up calling it.

I may need the files required for it to work out what the problem is.

yippiekayee

6th December 2008, 12:50

Alright here you go.

yippiekayee

6th December 2008, 14:00

Some good news and some bad news:
Jumper (US) and Independence Day (US) both decrypt fine and are byte identical with AnyDVD HD. And I redid Prison Break S3 Disc 4 and it is now okay as well.
However, the convtable generated by libblu-ray differs for ID4. And for Jumper, libbluray takes several minutes and then returns nothing at all.. there's also no output whatsoever on the commandline even with -v 64.

loo3aem3ON

6th December 2008, 15:00

I don't know which version of Jumper [...] - so I took my US copy of Jumper for a quick spin. The 0.1.3 debugger creates a convtable with no problems.. however the 0.1.4 debugger throws an ArrayIndexOutOfBoundsException.
That's because the subtable already existed with removed bogus descriptors. I use the length of the existing (newly created) subtable object to navigate to the next subtable. This will obviously fail if the new subtable is longer than the existing subtable object because it contains the bogus descriptor. Because the events and the conversion table are still not fully understood it is difficult to find a good implementation. Until recently we didn't even know these bogus descriptors existed.
Anyway i decided to postpone segment decryption/filtering until all parts of the conversion table have been received which should solve this problem. Here is the new official release: BDVM debugger 0.1.4 (http://uploaded.to/?id=7awa5o)

yippiekayee

6th December 2008, 15:31

The convtable also differs for Home of the Brave (US) and Ice Age (US) - both are properly being decrypted with the convtable from the debugger. The convtable also differs for Sunshine (DE).. but I haven't checked if the debugger convtable is correct.

dirio49

6th December 2008, 18:40

Is there any difference between contabd and contab? because in linux I only get contab.

yippiekayee

6th December 2008, 21:20

I also did the Simpsons Movie (DE).. same story with libbluray.. it creates a convtable but it's not identical with the one from the debugger.
And with Dr. No (US) the program just sits there for minutes maxing out one ofmy cores but otherwise doing nothing (no output, no convtable).
Since I haven't had one single success with lib-bluray I'm going to stop creating tables now.. I think there's something seriously wrong with the software at this point.. maybe it works on Linux but it definitely doesn't work properly on Windows.

loo3aem3ON

6th December 2008, 23:26

I also did the Simpsons Movie (DE).. same story with libbluray.. it creates a convtable but it's not identical with the one from the debugger.
Can you tell us what bytes differ? For example appending a few garbage bytes to a correct conversion table (fix-up table) doesn't make it invalid. If there is a problem with libblueray then we need to solve it.

Accident: Adding support to create diffarchive snapshots for the debugger is on my "ToDo" list. This will probably help you if you can't figure out the problems quickly. :)

yippiekayee

7th December 2008, 00:30

I don't really know how I could tell the difference other than ripping and md5sum'ing the results and that's just too time consuming - going through all movies once takes maybe 2.5 hours per disc and another rip would bring that up to 3.5+ hours. The best I can do is post the 6 convtables I got from libbluray and the corresponding debugger convtables (which are all verified against AnyDVD HD).

Accident

7th December 2008, 01:20

I can't guarantee the binary conv_tab.bin files would be identical. I will spend some more time with the Windows version, I need to implement the hash.db file first, since I can not fit 40GB files to try, (Windows does not do sparse files).

Other considerations is that it should work on big-endian, but has not been tested. It should work on 64bit, but has not been tested.

yippiekayee: I appreciate the hard work you have done, and I am committed to making it work for everything.

yippiekayee

7th December 2008, 02:00

Turns out Bender's Game is actually not using MKBv9 so that made the whole process of trying it out a little easier. Anyway, the convtable was created without error though the output of the debugger is quite different from any other movie.. and it is half the size of most other tables. ConvTableView has a lot of messages though like patch adresses being smaller than the previous one and patch exceeding TS packet boundaries. Decryption also went fine but about 2 minutes in (TMT wouldn't play the thing in movie mode, PowerDVD started using a lot of CPU power immediately and played choppy, and went to slideshow mode when I moved around the timeline) the picture started getting corrupted so I conclude that the movie wasn't properly decrypted.

My next stop will be Firefly but that is definitely MKBv9 so I'll have to go for a two step process.

@Accident: I'm on 32bit Windows (XP).

MickJT

7th December 2008, 04:39

My next stop will be Firefly but that is definitely MKBv9 so I'll have to go for a two step process.

Is this process removing AACS (only) with AnyDVD HD, and then BD+ with libbluray?

That won't necessarily work, as some of the newer BD+ require intact "with AACS" .m2ts files, and AACS and BD+ need to be removed at the same time.

KenD00

7th December 2008, 12:27

Thats not a problem right now because you can run the BDVM Debugger as stand alone application. The workflow is to use it on the still encrypted disc, and then decrypt the disc with DumpHD using the produced Conversion Table (or any other Ripper and applying the BD+ removal as second step).

:rolleyes:

Rupan

7th December 2008, 13:03

@Accident

Would it be helpful for me to test libbluray on 64-bit Linux?

EDIT: I've uploaded conversion tables produced on 64-bit Linux with libbluray and the bdvm java debugger. Both tables are exactly the same byte size but their contents differ greatly. I'm waiting on md5sum to finish, which may not be really done for a day or so, and will get back to you when it is complete.

These conversion tables were produced using Live Free or Die Hard (US).

http://www.sendspace.com/file/2s64n3

yippiekayee

7th December 2008, 17:35

Hmm.. so you actually decrypted the movie twice? md5suming the convtable should only take a second.. md5sum'ing a movie takes maybe 15 minutes or so if you have a decent harddisk.

I haven't managed to get DumpHD to fix BD+ on an AnyDVD HD decrypted disc - however seeing as the convtable is just as small as with Bender's Game. Once I know how to use DumpHD to remove the BD+ I'll check playback and report back.

yippiekayee

7th December 2008, 21:39

Thanks to the help of Kend00 I now managed to apply the convtable to Firefly.. and the resulting video stream is still corrupt.
There are two more discs in the set but I'm hoping that what I've posted is enough to improve the debugger to the point where it can handle the latest batch of Fox discs (I don't have any other titles and the list of titles AnyDVD HD cannot handle doesn't really include other titles that are interesting to me).

Rupan

7th December 2008, 23:21

Hmm.. so you actually decrypted the movie twice? md5suming the convtable should only take a second.. md5sum'ing a movie takes maybe 15 minutes or so if you have a decent harddisk.

I haven't managed to get DumpHD to fix BD+ on an AnyDVD HD decrypted disc - however seeing as the convtable is just as small as with Bender's Game and seeing the same events in the debugger logs I figured I'd post (http://rapidshare.com/files/171165285/Firefly-Disc1.7z.html) the BDSVM, hashes, contable and debugger output here just in case. Once I know how to use DumpHD to remove the BD+ I'll check playback and report back.

This is (again) a dump of the US version of Live Free or Die Hard. Any heavy file I/O operations on my workstation take 20-50% longer than on most other workstations. My disks are all backed with 256-bit AES encryption which consumes roughly 65% of one core during heavy I/O.

Here is a list of files that differ between the AnyDVD dump and the dump created by DumpHD+bdvmdbg. All of the video streams (m2ts) are byte identical:

./CERTIFICATE/app.discroot.crt
./BDMV/MovieObject.bdmv
./BDMV/PLAYLIST/* (about 90% of them anyway)
./BDMV/JAR/00000.jar
./BDMV/JAR/00002.jar

which I believe are related to the region locks and removal of annoying trailers. I assume the certificate differs because it contains cryptographic signatures of all the unencrypted files on the disc and must be fixed when their content is changed.

I'll re-rip with the convtable produced by libbluray and get back with the results later today.

EDIT2: I'm attaching a copy of the 64-bit convtab linux executable built from svn revision 158. My copy of Die Hard 4 is currently being processed by DumpHD 0.5.1 with the libbluray-produced convtable. When it is finished copying I will compare the checksums again and update this post.

Rupan

8th December 2008, 01:34

I just finished dumping and checksumming Die Hard 4 with the convtable produced by libbluray. All file checksums match the dump produced by bdvmdbg's convtable except the main title (33GB: 00000.mt2s). I suspect that this may be a bug in the libbluray implementation rather than a 64-bit issue since all the other file checksums match.

loo3aem3ON

8th December 2008, 01:44

Thank you for your support. Like i previously wrote i don't like to support more BD+ protected (damaged) movies than AnyDVD-HD and more importantly i don't want this project to make such ridiculous headlines again. Several sites were reporting of a break of BD+ which never occurred and now the vendor of my player seems to have stopped releasing updates. Being 1-2 months behind SlySoft is acceptable for me.
If you want to help this project then learn how BD+ works by reading the documentations and looking at the source code. I've also started this (https://forum.doom9.org/showthread.php?t=143247) thread where you can propose your ideas about how to exploit various weaknesses of BD+ which you might have found. Your ideas could lead to an attack which really breaks BD+ but i believe this is unlikely. Finally i also saw wikipedia reporting or "BD+ circumvention" which would mean repairing the damaged m2ts files without the execution of the content code. I don't know if this is possible in general but i am quite sure AnyDVD-HD executes the content code and so do we.

KenD00

8th December 2008, 03:51

All file checksums match the dump produced by bdvmdbg's convtable except the main title (33GB: 00000.mt2s)

Which is the only file that is BD+ protected...

:rolleyes:

Accident

8th December 2008, 05:13

Phew that is a lot of bad news coming my way. Can we sum up what is broken and needs fixing. Maybe also mention what works as encouragement... :)

I assume there are titles that:

* Do not work in Debugger, nor libbluray
* Work in Debugger but not libbluray
* Work in Unix but not Windows
* Work in 32-bit but not 64-bit.

Ice Age 1 MD5
Unix : e09cc8bf331b8283462c3b8ceebe889e
Win32 : e09cc8bf331b8283462c3b8ceebe889e
Jumper MD5
Unix : 9ce4443630cd7e5fce61ddeb84400fb8
Win32 : 9ce4443630cd7e5fce61ddeb84400fb8

(I take back my comment about sparse files under NTFS, works just fine).

I can not try PBS3D4, as it hashes file in AACS/

[trap] reading 'AACS/MKB_RO.inf' at pos 0000000000000000
[trap] reading 'STREAM/00000.m2ts' at pos 00000000e9c82600
[trap] reading 'STREAM/00000.m2ts' at pos 0000000098b5d600

yippiekayee:

Naturally, I do not expect you to post the entire file, but if you have time, run these commands, and post the files:

dd if=00000.m2ts of=shablock1.bin bs=512 skip=5004011 count=1
dd if=00000.m2ts of=shablock2.bin bs=512 skip=7660563 count=1

yippiekayee

8th December 2008, 09:46

Here's MKB.RO.info from Prison Break. 00000.m2ts is the only episode on that disc and it it 11GB so posting is out of the question both for bandwidth as well as legal reasons.
@rumpan: Do you have PUO/advertising/region code/BD Live removal active in AnyDVD? I turned all that off so that the output should match the output of DumpHD (although I have yet to hash non m2ts files.. but the Bond movies TMT doesn't play will be hashed soon since my AnyDVD HD rip plays in folder mode whereas the DumpHD rip doesn't).

Several sites were reporting of a break of BD+ which never occurred and now the vendor of my player seems to have stopped releasing updates. Being 1-2 months behind SlySoft is acceptable for me. What makes you think that this won't happen again regardless of whether you beat Slysoft to the punch? Sensationalist reporting is a fact of life... don't you recall the news about AACS being broken? I'd turn this whole argument around and say that the longer the current code stays unbroken and the more titles that are being released which cannot be copied, the more it strenghtens Fox in their belief that they shouldn't just give up on BD+ because the cost outweighs the benefits. If you think back to early this year, it was rumored that Fox was close to dumping BD and one of the reasons was cost.. with BD+ they have to swallow higher cost than other studios which are content with AACS.

Accident

9th December 2008, 02:10

loo3aem3ON: Clarify some of the recent changes for me please. You send events up to 50 "files", is the 50 just arbitrary, or a known upper-limit? In terms of program flow, do you only ever receive one conv_tab, or in fact, handle receiving multiples, and merge them in? Currently libbluray only receives and parses exactly one conv_tab. (Convert binary format to C structures)

Rupan:
Comparing the two conv_tab.bin, I find this to be interesting:

000006e0 00 00 50 3d 00 00 63 34 00 00 66 4e 00 00 6d bb #Deb
000006e0 00 00 50 3d 00 00 63 34 00 00 6d bb 00 00 75 38 #lib

000006f0 00 00 75 38 00 00 8b af 00 00 8e 2e 00 00 9a a9 #Deb
000006f0 00 00 8b af 00 00 8e 2e 00 00 9a a9 00 00 da 4d #lib

Debugger has an extra 4-byte value in there, otherwise the files match quite well. And a few more, we are off by about 36bytes after that index. I must be removing too many Type-2 descriptors.

yippiekayee: Don't miss my sha request in my previous post :)

loo3aem3ON

9th December 2008, 03:36

loo3aem3ON: Clarify some of the recent changes for me please. You send events up to 50 "files", is the 50 just arbitrary, or a known upper-limit?
The 50 is arbitrary. The right way to do it would probably be to read the filenames from the BDVM/STREAM/ directory and then calculate the parameter for event #0110 with some unknown mapping f(filenmame) = parameter. It's also the possible that the player gets this mapping from the content code. Maybe after event #0110 with last parameter = 0xFFFF?

In terms of program flow, do you only ever receive one conv_tab, or in fact, handle receiving multiples, and merge them in?
With those events #0110 (last parameter going up from 1 to 50) i announce the playback of hopefully all BD+ damaged files to get all the different subtables of the conversion table. Some conversion tables are bigger than 4MB so there is no way for the content code to pass such a big table to the player with a single call of TRAP_GetConversionTable. So you have to handle multiple calls of that trap and finally merge all the different subtables you've got into a big conversion table. I do this by creating a subtable object for each new subtable and after decryption/filtering i build a conversion table from all objects.

I must be removing too many Type-2 descriptors.
Yes. The first byte of the first valid descriptor you've mistakenly eliminated is 0xB6.

yippiekayee: Don't miss my sha request in my previous post :)
You really should add support for this simple hash database format. If you can't open the file you query this database. Don't hesitate to ask me if you have any questions about it.

Rupan

9th December 2008, 03:42

loo3aem3ON: my apologies, but I don't think that I am qualified to participate in development here. I can code in C and Java just fine, but when it comes to math-based security I just get lost. I averaged a B in calculus I through III and hated every minute of it. FYI, I did try to read your BD+ documentation and failed to grasp it to a point where I can help.

Accident: its good that what work I can do is helping.

Given the above, I'd be glad to help with code that doesn't involve crypto so let me know what I can do.

Accident

9th December 2008, 06:35

The 50 is arbitrary. It's also the possible that the player gets this mapping from the content code. Maybe after event #0110 with last parameter = 0xFFFF?

Yes, I had noticed that some BD+ disks does do that call. If I find time, maybe I'll check to see if I happen to find the right number.

So you have to handle multiple calls of that trap and finally merge all the different subtables you've got into a big conversion table.

I suspected as much! I will modify the sources to handle being called multiple times, and either add-with-sort, or postpend, and sort later.

If a given title has multiple subtable, and you are only given one decrypt key, I assume you keep the key, or ask for it multiple times? Do we have the BDSVM where this is required that I can test with?

Yes. The first byte of the first valid descriptor you've mistakenly eliminated is 0xB6.

I thought everyone knew that 0xB6 is evil.

You really should add support for this simple hash database format. If you can't open the file you query this database. Don't hesitate to ask me if you have any questions about it.
Naturally, and it won't be a problem, but I didn't think we had hashdb file for PrisonBreak anyway, so I can't test it either way.

Checking through the code I find:

if (entry->flags & 0x80) {

Terrible! I shall fix this asap. I am definitely removing too many repair descriptors.

Edit:

Now I'm starting to second-guess myself. Technically the &0x80 test is wrong, since it would include "type 3" descriptors in what is being removed, but 0xB6 is a valid type 2 ? So it would then presumably be the bit check after...

bits = entry->flags & 0x3f; // 6 bits, 0-63
if (!(mask[ (bits >> 3) ] & (1<<(bits & 0x07)))) {

..that is incorrect.

Edit2: Comparing against Debugger, I see you mark them inactive if type is 0 and 3, and well as using 7-byteoffset. I have done the same sort of changes, and we should retry against DH4.

Edit3:

Rupan:

That is excellent, thank you for your assistance.

Rupan

9th December 2008, 11:49

The updates you pushed out in svn revision 159 fixed DH4. The conversion tables produced by libbluray and bdvmdbg are byte identical. Good work!

I've attached a 64-bit build to this post in case anyone wants to try it out.

yippiekayee

9th December 2008, 19:15

Don't miss my sha request in my previous post I didn't really get what you wanted but I attached the mkb_ro file to my post but for some reason it never showed up - and my post suddenly appears to be edited by a mod.
Either way.. I'm attaching the hash_db file created by the debugger and here I ran sha1sum on a few files:
3976563d407eb565a822dfdc5bfad4bb5c299eff 00000.m2ts
da9af84a1221f06b66178fd3d1c3cb8e35d93538 Content000.cer
ea9b1b1ba6bd7d3cdf8d03e146b0cb891dfa8bbb ContentHash000.tbl
ffdfb3d33001abaadded8ac0ea4a447b6347186f ContentRevocation.lst
5cf9723d9e870514adb31ae8f7cd1dd591c1fa5d CPSUnit00001.cci
28357c70a24094c8bdff90acfd886cec0f015dad MKB_RO.inf
285496166a6bad574f49916cb2da4edc8cdb90fd MKB_RW.inf

dirio49

9th December 2008, 20:10

@yippiekayee
the dd command is a linux command, to read in raw mode (unless i am mistaken)
this link is for the windows executable
http://www.chrysocome.net/downloads/dd-0.5.zip

site http://www.chrysocome.net/dd

hope this helps

loo3aem3ON

9th December 2008, 20:45

Manually extracting blocks from arbitrary files on the disc is no acceptable long term solution. You should use the hash database which is easy to implement and automatically generated.

Accident: You can find an example in the repository now. (direct link) (https://trac.assembla.com/bdplus/browser/trunk/doc/debugging/hash_db.txt)

Guest

9th December 2008, 21:54

I attached the mkb_ro file to my post but for some reason it never showed up - and my post suddenly appears to be edited by a mod. The file appeared to be a complete intact file from a DVD. We can't post copyrighted things like that.

Accident

10th December 2008, 01:12

Manually extracting blocks from arbitrary files on the disc is no acceptable long term solution. You should use the hash database which is easy to implement and automatically generated.

At the time, we had neither file format. "dd" just meant anyone could run it, on any OS, without needing to setup java, or run debugger. But, since he already had the hashdb.bin file, even better. I have grabbed it, and implementing hashdb support now.

Edit:

The hash_db.bin from yippiekayee, which I think is for PBS3D4, runs like this:

[diff] Attempting to open 'hash_db.bin' looking for 'AACS/MKB_RO.inf'
[diff] namehash: 00 00 00 00 00 00 00 00 00 00 02 00 41 41 43 53 2F 4D 4B 42 5F 52 4F 2E 69 6E 66
[diff] find hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] found digest, reading 00000014 (20) bytes...
[diff] Attempting to open 'hash_db.bin' looking for 'STREAM/00000.m2ts'
[diff] namehash: 00 00 00 00 E9 C8 26 00 00 00 02 00 53 54 52 45 41 4D 2F 30 30 30 30 30 2E 6D 32 74 73
[diff] find hashdb: 93 C4 D2 BB 20 66 51 56 AB C7 B4 DE 6C 19 58 F7 5A 29 0D 0F
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D
[diff] Attempting to open 'hash_db.bin' looking for 'STREAM/00000.m2ts'
[diff] namehash: 00 00 00 00 98 B5 D6 00 00 00 02 00 53 54 52 45 41 4D 2F 30 30 30 30 30 2E 6D 32 74 73
[diff] find hashdb: C7 B2 99 A3 91 7E 9C AF 2C D6 62 3C 07 A9 0C 6D 01 25 9C 33
[diff] read hashdb: 3C D4 3F C6 32 99 69 FE 19 DE BB 7E E5 9C E3 8B BA F4 90 77
[diff] read hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] read hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D

So it successfully finds the first entry, which makes me believe my hash_db.bin code is correct, but fails to find the other two. Hexdump confirms there are only 3 entries in that hash_db.bin.

loo3aem3ON: You may want to throw in a comment in hash_db.bin documentation not to SHA hash the C-string NULL terminator byte for the filename. I guess as much, but I did try both just to be safe. :) Although your examples clearly show you not including them.

Edit2:

Oh I see, some file-paths are inside of BDMV/ and some are full-path. Quite annoying that. However, now I get:

[diff] Attempting to open 'hash_db.bin' looking for 'AACS/MKB_RO.inf'
[diff] find hashdb: 41 43 7C B2 75 82 54 C4 53 94 DD EA F6 F3 26 67 E1 66 EE 12
[diff] found digest, reading 00000014 (20) bytes...
[diff] Attempting to open 'hash_db.bin' looking for 'BDMV/STREAM/00000.m2ts'
[diff] find hashdb: A1 68 3D F0 79 BF 6D 66 F6 CD B4 2A DC D4 00 8A 76 43 7A 9D
[diff] found digest, reading 00000014 (20) bytes...
[segment] Starting decode of conv_tab.bin: 0x4dfafc (1044836)
[segment] num tables 79

[segment] Key 0, 0: 9CF737C8A6E37D314472789794CC19A7 mask:326FC93CE78C0C6A

[segment] Key 0, 7: 20AB48B7B57FD04DE46E81855DCD8E65 mask:FFFFFFFFFFEFFFFF
[segment] removing entry 121 (flags 80: bits 20 => byte 5, set 10 to false)

MD5 (PBS3D4/conv_tab.bin) = 694e63c6e486231659c63619fa381b98

Which might just be the correct result.

loo3aem3ON

10th December 2008, 15:33

loo3aem3ON: You may want to throw in a comment in hash_db.bin documentation not to SHA hash the C-string NULL terminator byte for the filename. I guess as much, but I did try both just to be safe. :) Although your examples clearly show you not including them.

Edit2:

Oh I see, some file-paths are inside of BDMV/ and some are full-path. Quite annoying that. However, now I get:
The filename is exactly the string from the vm memory:RAP_MediaSHAFileHash(UINT8 *FileName, UINT32 FileNameLen, UINT32 FileOffsetHigh, UINT32 FileOffsetLow, UINT32 *len, UINT8 *dst)
That is <FileNameLen> bytes from address <FileName>.

Nice to see you got it working. Do you also return <len'> ?

Given the above, I'd be glad to help with code that doesn't involve crypto so let me know what I can do.
You can improve the source code of the debugger. I don't have much experience programming in java. :rolleyes:
BD+ is basically an arbitrary program running on a virtual DLX like processor. This program executes some tests to check if it runs on a valid player (trusted environment) and if all tests passed it generates the conversion table (fix-up table) which is then used to repair the damaged m2ts files. Some tests involve trap calls which do cryptography but they make up only a very small part of the source code. ;)

Accident

11th December 2008, 04:57

Yep, I do set len', as in "the len of sha blocks we actually managed to read".

So, to return to the multiple conv_tab blocks, which BDSVM files has/needs this so I can test with it?

loo3aem3ON

11th December 2008, 20:36

Below is a development snapshot of the upcoming version 0.1.5 of the debugger. This version supports (diffarchive) snapshot recording. Instead of pressing "run" you need to select "snapshots" -> "Record snapshots" from the menu and then wait for about 30min (depending on your pc) for this process to complete. The snapshot archives are stored "snapshots".

BDVM debugger 0.1.5 (dev) (http://uploaded.to/?id=jf0qm6)

For those discs from which you couldn't get any conversion table with libblueray but the debugger seems to execute everything properly please record snapshots. Please upload the content of the "BDSVM directory from the disc as well as the contents of the "snapshots" directory the debugger created after recording.

Accident: I have switched everything (raw diff segments as well as the diffarchive metadata) to BIG endian as requested. All future snapshot packages will be big endian. I have already converted the jumper package to big endian. :)

So, to return to the multiple conv_tab blocks, which BDSVM files has/needs this so I can test with it?
Yippiekayee reported the problem in posting #506 (https://forum.doom9.org/showpost.php?p=1220969&postcount=506) which also contains the link to the content code. My reply in posting #511 (https://forum.doom9.org/showpost.php?p=1221095&postcount=511) gives a short explanation of the problem.

Accident

12th December 2008, 03:44

Accident: I have switched everything (raw diff segments as well as the diffarchive metadata) to BIG endian as requested. All future snapshot packages will be big endian. I have already converted the jumper package to big endian. :)

That was surprising, and thanks for the warning. I will keep that in mind when I next get snapshots.

Yippiekayee reported the problem in posting #506 (https://forum.doom9.org/showpost.php?p=1220969&postcount=506) which also contains the link to the content code.

The hash_db.bin file appears to be in the previous hash_db format. I don't suppose anyone has the file in the current format? Guess I could temporarily add support for the old format as well.

Edit: Not to worry, I have hardcoded it for now, so I can test it against multiple conv_tab entries.

Edit 2:

Like that of Debugger, I post 110,0,1 followed by 110,0,2 .... 110,0,24. I receive a non-zero conv_tab for both "fileNumber" 1, and 2. The rest result in 0 length conv_tab. This makes me think there are two parts of me to get. But what I can not quite figure out, is that both conv_tab that I receive are mostly the same. I save the data from 110,0,1 and 110,0,2, do a binary compare, and they are virtually identical. There are less than 8 bytes that differ (roughly in the middle of the file). I am unsure what sort of "merge" I should do with multiple conv_tables.

loo3aem3ON

12th December 2008, 09:30

Like that of Debugger, I post 110,0,1 followed by 110,0,2 .... 110,0,24. I receive a non-zero conv_tab for both "fileNumber" 1, and 2. The rest result in 0 length conv_tab. This makes me think there are two parts of me to get. But what I can not quite figure out, is that both conv_tab that I receive are mostly the same. I save the data from 110,0,1 and 110,0,2, do a binary compare, and they are virtually identical. There are less than 8 bytes that differ (roughly in the middle of the file). I am unsure what sort of "merge" I should do with multiple conv_tables.
I have never compared them and i am surprised they differ. If you happen to have both versions on disc could you please upload them so i can take a look? The way i handle it currently is creating a subtable object for every new subtable i get. If an object with that subtable id already exists i ignore it. Therefor in this example all subtables which come with the second TRAP_GetConversionTable call are ignored.

I really need to figure out the meaning of the second parameter of event#0110. My closest guess was that the m2ts are enumerated 1...[number of files]. Example:
filename | id
---------------------
00000.m2ts | 1
00002.m2ts | 2
00004.m2ts | 3
00006.m2ts | 4
00008.m2ts | 5
00010.m2ts | 6
00012.m2ts | 7
00014.m2ts | 8
00016.m2ts | 9
00018.m2ts | 10
.... | ...
So when you start the playback of 00012.m2ts you would create an event 0110,0,7. More tests are needed to confirm/disprove this.

Accident

12th December 2008, 09:43

Certainly. jumper_raw_tables.7z (http://uploaded.to/?id=us4cw5). These are saved on trap_ConvertionTable() so they have not been touched by me at all. Alas, this includes decoding.

The differences are:

00100180 00 00 00 00 00 03 b5 c0 00 06 24 10 00 3d fc c4 |......��..$..=��|
00100180 00 00 00 00 00 03 b5 f8 00 06 24 10 00 3d fc c4 |......��..$..=��|

00100200 00 06 29 8c 00 3d fd 38 00 02 e4 c4 00 03 b5 c0 |..)..=�8..��..��|
00100200 00 06 29 8c 00 3d fd 38 00 02 e4 c4 00 03 b5 f8 |..)..=�8..��..��|

00100250 00 02 50 f8 00 3d fd 78 00 00 00 00 00 00 00 10 |..P�.=�x........|
00100250 00 02 50 f8 00 3d fd 78 00 00 00 00 00 00 00 0b |..P�.=�x........|

00100260 00 00 00 70 33 92 f8 11 00 02 ea 10 00 1e ff fc |...p3.�...�...��|
00100260 00 00 00 87 39 07 9b c3 00 02 ea 10 00 1e ff fc |....9..�..�...��|

00100280 00 00 23 80 00 00 00 00 00 3d fd ac 00 02 ef 24 |..#......=�..�$|
00100280 00 00 23 cc 00 00 00 00 00 3d fd ac 00 02 ef 24 |..#�.....=�..�$|

I will leave my conv_tab code untouched for now. If I get one, that is the one I deal with.

I would have expected to use "tableID" somewhere as well (the tableID stored in conv_tab), but so far, I have not.

loo3aem3ON

12th December 2008, 17:37

Thanks.

Were you able to find out what TRAP_#8020 does? I would like to implement it. (see posting #507 (https://forum.doom9.org/showpost.php?p=1220977&postcount=507)).

I will leave my conv_tab code untouched for now. If I get one, that is the one I deal with.
The conversion table may be split. In theory you won't get the complete conversion table unless you announce the playback of all files. Please read again postings #461 (https://forum.doom9.org/showpost.php?p=1218518&postcount=461), #471 (https://forum.doom9.org/showpost.php?p=1218761&postcount=471) and #479 (https://forum.doom9.org/showpost.php?p=1218870&postcount=479). I'll investigate the mapping [file number] -> [second parameter of event#0110] soon. Currently i'm looking at the data returned by the callbacks/events.

yippiekayee

12th December 2008, 18:20

@Accident: are you working on the US version of Jumper? And since Rupan had better success with the latest code could you post the latest libbluray for windows?
I did not keep the md5sums or ripped files of my previous discs but I kept the convtable of every successful rip so once you think you're ready for some more testing I could run a few discs and give you the debugger and libbluray output for comparison.

For those discs from which you couldn't get any conversion table with libblueray but the debugger seems to execute everything properly please record snapshots. Please upload the content of the "BDSVM directory from the disc as well as the contents of the "snapshots" directory the debugger created after recording.I take it that's mean for me right? So for now I think I'm supposed to run this on Dr. No.
I'm afraid I'm not much of a math buff either.. I suffered through cryptography in college and was glad when it was over.. same story with the majority of other math classes I had to take.. I failed almost no classes but the three I did fail all were pure math (at least according to my definition.. some would argue it's applied math but to me it's all the same). I don't mind ripping discs or creating those snapshots to help out, but as far as attacking BD+ or making this thing work with the latest BD+ titles goes, I don't think I'm your guy.

I just got 4 new BD+ discs

Accident

13th December 2008, 01:21

I was going to push out a new version once I had worked out what to do with split conv_tabs, but since I am still unsure about that, I might as well push it out so we can test it more.

libbluray-0.0.7.tar.gz (http://uploaded.to/?id=w8a5g7)

yippiekayee

13th December 2008, 14:05

I'm afraid libbluray-0.0.7 is no good for me.. tried running it on Dr. No (one of the titles where 0.0.6 didn't return a convtable) and it crashes after a few seconds. I'm now creating snapshots with the new debugger.

yippiekayee

13th December 2008, 15:34

libbluray also crashes on Live And Let Die.. whereas the 0.06 version just keeps on running indefinitely. Stargate Continuum runs for about 30 seconds but there's no output.. so I'm creating diffarchives as well.

The debugger also has a non fatal bug: if there's no hash_db.bin there's an uncaught FileNotFoundException... just wrap the file loading attempt into a try/catch and that won't happen again.

yippiekayee

13th December 2008, 17:21

Here's an idea to make it easier for people to help: The debugger should have a function to zip up everything needed for you guys to reproduce... volume id, hashes database, bdsvm folder and snapshots - or at the very least copy all this to one folder which can be zipped up and uploaded somewhere. And if there's every any other files needed, those could be added as well so that whomever posts a problem could immediately post all the files you guys need to reproduce as well thus making the process of error reporting a lot more efficient.

@loo3aem3ON: is there any point to me creating snapshots for the latest BD+ titles that the debugger can currently not handle properly?

Also, check your PMs..

loo3aem3ON

13th December 2008, 22:02

I discovered that event#0000 starts the virtual machine. So when we clear the first 1000h bytes and then start execution at 1000h we actually call the handler for event#0000 :D
We already knew that event#0010 shuts the vm down so the event/callback ids seem to be grouped too. Those two belong to group 0 obviously. I am still tracing the second parameter of event#0110 backwards to it's source. So far i've figured out that it's a 16-bit value. I actually wanted to take a small break now but i got so many new messages...

The debugger also has a non fatal bug: if there's no hash_db.bin there's an uncaught FileNotFoundException... just wrap the file loading attempt into a try/catch and that won't happen again.
Yes, you should not delete this file. I will create a new (empty) file in case it is missing.

Here's an idea to make it easier for people to help: The debugger should have a function to zip up everything needed for you guys to reproduce... volume id, hashes database, bdsvm folder and snapshots - or at the very least copy all this to one folder which can be zipped up and uploaded somewhere. [...]
I will think about that.

@loo3aem3ON: is there any point to me creating snapshots for the latest BD+ titles that the debugger can currently not handle properly?
No there is no point doing that. Only create snapshots for those who work correctly.

Also, check your PMs..
Please don't send me the snapshots. Accident is the one who needs them. ;)

Also please don't spam the thread with unnecessary postings. If you made the last posting in the thread you should edit it to add more comments.

Accident

14th December 2008, 02:56

Slight bug left from trying to merge convtabs meant 0.0.7 was very broken.

libbluray-0.0.8.tar.gz (http://uploaded.to/?id=k8gjl8)

Rupan

14th December 2008, 06:19

I purchased The Simpsons Movie to help debug the VM. It produces a conversion table which, when used with DumpHD, produces identical output to AnyDVD. The VM had some interesting output this time:

[Event #00000001] EVENT_0210( 00000000, 00000001 )
[W] TRAP_DeviceAccess not implemented!

Is there any documentation available for event 210 and/or DeviceAccess? Depending on what they do I may be able to help. Just a pointer to the document that I should read would suffice.

@Accident: I've produced a diffarchive snapshot for The Simpsons Movie and will upload it later tonight. Look in your PMs for the link.

yippiekayee

14th December 2008, 10:45

Slight bug left from trying to merge convtabs meant 0.0.7 was very broken.Does that mean that convtable merging should now be working (and thus this version would be ready for Prison Break)?

And as far as holding back and letting Slysoft take the lead - that just backfired (http://it.slashdot.org/article.pl?sid=08/12/13/137257). This will make handling of the latest batch of BD+ titles a much bigger story than it ever could've been if they'd be handled by now. It is generally acknowledged that it might take some time to catch up with copy protection.. but take too long and it will be chalked up as a (temporary) win for the studios and that makes it a newsworthy story. It also made it up to engadget so I figure it's just a matter of time until it's up on ars and other more mainstream tech news outlets.

Accident

14th December 2008, 12:00

I don't know. The version of Jumper that was supposed to produce multiple conv_tabs, just produced two (almost) identical files, so there was nothing to be gained there. If the Prison Break you mentioned is PBS3D4 files, then it already produces a conv_tab (libbluray-0.0.8 version). I also grabbed the first 5 files releases sent in PM, and they all produce conv_tabs. Do we know of any releases that are still broken with libbluray-0.0.8 ?

-rw-r--r-- 1 owner group 1044836 Dec 10 14:23 PBS3D4-conv_tab.bin

yippiekayee

14th December 2008, 12:31

Do we know of any releases that are still broken with libbluray-0.0.8 ?I'll have to check. I hope it is okay if I get you the debugger and the libbluray convtables as I didn't keep md5sums (plus even though DumpHD is considerably faster than AnyDVD HD it takes a long time to rip a title) - and so far the output of the two applications has always differed.
And I mostly meant the first three discs of Prison break - they produce significantly larger convtables (4 MB) so I figure that would be the real test.
Did you compare the convtables I already sent you (you should have the Prison Break Disc4 from the debugger if I'm not mistaken) with the one you created on your own? Do you know if they produce the same outcome?

Accident

14th December 2008, 13:35

I merely confirmed they produced tables that "look good". I will compare against the actual files. I would love the svm files for a release that produces the 4mb tables, since that would have to use multiples.

loo3aem3ON

14th December 2008, 13:43

It is generally acknowledged that it might take some time to catch up with copy protection.. but take too long and it will be chalked up as a (temporary) win for the studios and that makes it a newsworthy story. It also made it up to engadget so I figure it's just a matter of time until it's up on ars and other more mainstream tech news outlets.
If have no doubt that SlySoft could support the latest batch of BD+ damaged disc (and so do we). If we go forward and correct the bugs which prevents the new content code generation to execute properly the DRM industry will go after us. And this is something SlySoft probably wants to see. So we are holding back bugfixes like SlySoft does. Final decision.

yippiekayee

14th December 2008, 14:57

That's your prerogative, however I feel compelled to point out some obvious flaws in your argument:
The number of titles that can be handled versus those that cannot is much larger.. so why did Macrovision sit on their collective asses and let Slysoft handle BD+ titles since November 20th 2007 and strip BD+ since March 19th 2008? Why would they suddenly change their minds and get the lawyers involved just now and wouldn't they face the same situation when they finally update their BD+ handling code regardless of when they do that? And on the other hand they have a number of paying customers obviously waiting for the release which handles all the latest titles - so delaying something to let somebody else have first crack doesn't make a lot of sense to me. January 1st I'd understand because of the subscription model (and who knows, maybe it'll happen then), but why February?

And the other part of it is that since I've been the one person (now joined by Rupan) who listened to your call for support and invested quite a bit of time (I have no illusions that it's nowhere near what you guys invested) and money (you have a list of my titles.. about half of it I got specifically to help out here and not because I wanted to get them) I cannot help but feel a bit betrayed and ask why I should bother going on if you're not willing to. If the titles that can be handled right now are all that are ever going to be handled then I guess I could just as well sit back and wait for Slysoft since I do have a license and who cares about those who cannot or don't want to use AnyDVD HD.

I realize you don't owe anybody anything, but that cuts both ways and at that point I feel a bit disillusioned about what I've been doing since I've started posting here.

loo3aem3ON

14th December 2008, 15:46

That's your prerogative, however I feel compelled to point out some obvious flaws in your argument:
The number of titles that can be handled versus those that cannot is much larger.. so why did Macrovision sit on their collective asses and let Slysoft handle BD+ titles since November 20th 2007 and strip BD+ since March 19th 2008?
The industry is probably unable to sue SlySoft because it is legal on the small island they are on.

Why would they suddenly change their minds and get the lawyers involved just now and wouldn't they face the same situation when they finally update their BD+ handling code regardless of when they do that?
You yourself sent the links to the recorded snapshots as private messages in fear of prosecution. Avoiding prosecution is everyones top priority.
I must admit i don't quite understand your question. The execution environment is not changed (there is only one version of BD+) but the content code running in this virtual machine can be unique for each disc. Bugs in our implementations are exploited to distinguish between a licensed player and an unlicensed emulator.

And on the other hand they have a number of paying customers obviously waiting for the release which handles all the latest titles - so delaying something to let somebody else have first crack doesn't make a lot of sense to me.
The open source tools can become a threat to the business model of SlySoft. The open source tools aim for blue ray playback in linux without proprietary software so the users will probably accept if the latest movies are not supported.

I realize you don't owe anybody anything, but that cuts both ways and at that point I feel a bit disillusioned about what I've been doing since I've started posting here.
Without any doubt you have helped the project. One of the major problems you've helped correcting was the handling of multiple calls of TRAP_GetConversionTable which each time gives another part of a larger conversion table (fix-up table). Libblueray seems to have some other problems too but your snapshots will probably help fixing those issues.

SlySoft won't delay the bugfixes forever or otherwise their customers will lose faith in this company. Therefor there is no reason to believe that those movies you have had problems with will never be supported. Finally I told you very early that i don't want our implementations to compete with SlySoft's AnyDVD-HD (see posting #490 (https://forum.doom9.org/showpost.php?p=1219291&postcount=490)) but you kept on going.

zeroprobe

14th December 2008, 16:46

Is it possible to have some sort of brief layman term update of what the status is with the bd+ decryption here?

And what happened to Oopho2ei if you don't mind me asking?

LoRd_MuldeR

14th December 2008, 17:16

And what happened to Oopho2ei if you don't mind me asking?

He stopped his work on breaking BD+ (in the public) because he fears the consequences, now that his work became famous on the net. The AACS LA will certainly try to find out his identity to put a lawsuit on him (if not worse). So consequently he tries to protect his identity and his life now. Seems his Doom9 account has been deleted too.

yippiekayee

14th December 2008, 17:23

You yourself sent the links to the recorded snapshots as private messages in fear of prosecution.Prosecution isn't the right term.. as you know neuron2 objected to an attachmend I made and when I inquired on what is okay and what isn't (this board has strict policies and I don't want to get myself or the board operators into trouble) and so unless the admin gives different directions on that issue, I figured it would be prudent to refrain from posting any content files.
The open source tools can become a threat to the business model of SlySoft.Umm.. hasn't backuphddvd, backupbluray, dumphd, etc. always been the competition and aren't all of those tools running on Windows? In fact, it was the community here that came up with the first tools to circumvent AACS when Slysoft had nothing in their hand.
And, Slysoft's ripper was an afterthought.. it all started out with on-the-fly decryption so I figure on-the-fly-decryption is the main market for Slysoft. Just look at how complex the whole process of backup up a Blu-ray disc still is.. it is way beyond the capabilities of most (not on this board but most people aren't as knowledgeable). And even prior to HD there were plenty of tools who at least on a ripper level (there hardly is any on-the-fly decryption tool for DVDs) competed (and those that are still updated still do compete) with Slysoft's offering. Take DVD Fab HD Decrypter for instance.. it is probably AnyDVD ripper's number one contestant and it is free. And the demise of DVD Decrypter and RI4M had nothing to do with Slysoft either (check the Doom9 news for details).

Anyway.. I guess we can argue this all day long and neither one of us is going to change their mind.. so I'll just finish the things I'm working on for Accident and then retire.

@zeroprobe: It seems the debugger can handle any titles except those that are on Slysoft's list of currently unsupported BD+ titles (http://forum.slysoft.com/showthread.php?t=21985) - and that includes any future titles that might be added to that list. If you read through this thread (the last few pages) you'll notice a number of titles that are confirmed to be handled correctly.

loo3aem3ON

14th December 2008, 17:24

Is it possible to have some sort of brief layman term update of what the status is with the bd+ decryption here?
First of all BD+ doesn't do any decryption of the movie content. Instead BD+ protection means that your *.m2ts files are damaged after AACS decryption. To get the repair instructions you have to execute the program on the disc called "content code". The execution takes place inside a virtual machine which basically is a DLX like processor with a more complex player interface. The content code runs a series of tests to distinguish a licensed player from an unlicensed emulator. If one of these tests fails this content code won't give you the repair instruction (at least not without manual intervention).
Now to your question: the content code shipped with the newer discs usually runs more tests and evaluates the results more strictly. This way new bugs in our emulators show up which have to be fixed.

And what happened to Oopho2ei if you don't mind me asking?
Nothing has happened to him. Actually neither AACS nor BD+ has been broken and if these tools are only used to playback the contents and not to make the copy how can this be called a break of copy protection?

zeroprobe

14th December 2008, 18:21

Nothing has happened to him. Actually neither AACS nor BD+ has been broken and if these tools are only used to playback the contents and not to make the copy how can this be called a break of copy protection?

I just remember he was a big part of getting this going. So as I understand progress is on pause for now?

Could you guys if you wanted, overcome the BD+ on the newest of discs? I'm just curious how far you guys could go, or are these new revisions alot more complicated like what Slysoft have said?

Wilbert

14th December 2008, 20:03

Your questions have been answered a dozen times (implicitly) in this thread, so i don't understand why you ask them again.

loo3aem3ON

14th December 2008, 22:54

yippiekayee reported a bug which occurs when the debugger is started without the hash_db.bin present. Are there any more problems? Are the snapshots created properly? Accident, have you had any problems with the new snapshots created by the debugger?

I would like to release version 0.1.5 soon because KenD00 has been working on dumpHD to load the debugger as a library and this needs some testing. On the other hand i would rather not release 0.1.5 if there are major problems with the recent development snapshots (0.1.5 dev).

Btw. Rupan has joined our development team. I hope he can help us improving the debugger. :)

KenD00

14th December 2008, 23:25

What yippiekayee reported is not a bug but more a cosmetical problem. Before an entry gets written it is tried to load the same one too see if its already present, this causes the exception when the file is missing, but it is caught and the code behaves properly, its just the stack trace that gets printed out.

:rolleyes:

Accident

15th December 2008, 02:02

Accident, have you had any problems with the new snapshots created by the debugger?

Sorry, I'm still catching up on the 15 titles or so I received, so I've not actually got to the stage of trying the new diffarchives.

Do not hold back on account of me though.

I have received an example of multiple conv_tabs (PB3S1) so I will work on merging conv_tabs, then push out 0.0.9.

Edit:

Oh I see, I cheated and just kept the "offsets" to write out, but when merging conv_tab, that clearly will not work. So it was writing 1MB data in place multiple times. I need just to re-compute the correct offset and we should be set.

-rw-rw-rw- 1 4136246 Dec 3 03:30 conv_tab-prisonbreak_s3d1.bin
-rw-r--r-- 1 4136246 Dec 15 11:11 new_conv_tab.bin

Edit 2:

I compute the new offset values, and I think the new conv_tab.bin are correct. They differ with the Debugger, but it appears to be merely an ordering. At offset 001f51c0 the Debugger writes tableID 00000002, whereas libbluray writes tableID 00000005. libbluray writes tableID in order it receives it, whereas Debugger appears to sort per tableID. Apart from that, I would like to think it handles multiple conv_tabs.

Edit 3:

Quick call to qsort() and diff(1) confirms both conv_tabs are identical.

Edit 4:

libbluray-0.0.9.tar.gz (http://uploaded.to/?id=a2tcyn)

Peer van Heuen

15th December 2008, 09:20

If have no doubt that SlySoft could support the latest batch of BD+ damaged disc (and so do we). If we go forward and correct the bugs which prevents the new content code generation to execute properly the DRM industry will go after us. And this is something SlySoft probably wants to see. So we are holding back bugfixes like SlySoft does. Final decision.

Now let me please put a stopper on this, before it gets out of control.

We are not the mafia.

By no means would we ever use tactical measures to watch any of you get into trouble. That would be the most shabby thing to do and silly and ineffective just as much.
We don't even wish that anything of this sort happens to our commercial competitors.

Also: we are not holding anything back.
Neither are you.

I know this whole DRM-related business is a hotbed for conspiracy theories - and up to a certain point I enjoy reading them...

yippiekayee

15th December 2008, 10:16

Good news.. I just ran the new libbluray on the entire Prison Break series and it produced byte identical output do the debugger.
I also ran it again on Independency Day in the hopes that now all tables created would be the same.. but that's not the case. I'm hoping that with the convtables I've sent you from both the debugger and libbluray you can determine if they'd produce the same output - perhaps some convtable analyzation tool would be in order so that anybody could test their convtables on their own and just report back any problems and which titles work okay without having to rip and compare against AnyDVD HD.

loo3aem3ON

15th December 2008, 11:04

[Event #00000001] EVENT_0210( 00000000, 00000001 )
[W] TRAP_DeviceAccess not implemented!

Is there any documentation available for event 210 and/or DeviceAccess? Depending on what they do I may be able to help. Just a pointer to the document that I should read would suffice.
Have a look here (http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220070033419%22.PGNR.&OS=DN/20070033419&RS=DN/20070033419).
TRAP_DeviceAccess sends commands to a device within the player and TRAP_DeviceDiscovery receives data from a device.

Accident

15th December 2008, 12:30

Ah, I don't seem to have ID4 BDSVM files anywhere. Is there a link somewhere? I have got everything sent to me in PM btw.

loo3aem3ON

15th December 2008, 18:11

I think (hope) i am getting close to the source of the second parameter of event#0110.

Anyway during a short break i found a comment of "Nate Lawson" who co-designed BD+ here (http://rdist.root.org/2008/04/11/designing-and-attacking-drm-talk-slides/):
Modern DVD and BD replication equipment can write a unique number on each disc (called burst cutting area on DVD, PMSN on Blu-ray). The entire disc is still the same but each one is slightly personalized.

When you write the BD+ code, you can put some “poison pill” logic in as well. For example, it might be “run SHA1(readPMSN(), secretValue) and if it equals magic value 87B2A058…, stop playback
Anyone knows about this PMSN? How can i read this number?
This "readPMSN()" call is probably TRAP_DeviceDiscovery with some special parameters. Maybe it also requires TRAP_DeviceAccess to send some low level atapi/scsi commands.

Edit:
The Pre-Recorded Media Serial Number is optional for BD-ROM disc. For BD-ROM, the Pre-Recorded Media Serial Number shall be stored in the BCA record of the BD-ROM disc.

xkodi

15th December 2008, 19:07

Anyone knows about this PMSN? How can i read this number?

Burst Cutting Area is used to store Media ID for CPPM/CPRM protected media like DVD-Audio discs:

http://www.geocities.com/columbiaisa/dvd_copy_protect.htm :

CPRM is similar to CPPM in using a MKB. All media used with CPRM must include a BCA (Burst Cutting Area) which can be used to record a unique media ID as part of the manufacturing process. This will then prevent the data from being copied to another disc because it will not have the same unique media ID and therefore the data cannot be decrypted.

Burst Cutting Area can be read under Windows with the functions defined in ntddscsi.h header file, which allow you to access SCSI port adapters. you should be able to use ntddscsi.h functions with every drive (IDE, SATA) under Windows, because they are emulated as SCSI.

i'm sending you a PM with example how to use ntddscsi.h header file to read BCA on DVD. i hope it will help you to do the same thing with PMSN.

derbeDeus

15th December 2008, 19:21

Anyone knows about this PMSN? How can i read this number?

PMSN is for online stuff to provide additional services. As the docs say, it's unique per disc copy. So the only use in BD+ would be to ban a certain disc from a naughty user :sly:

btw: it was not available on the disc i tried (i got a scsi err)

KenD00

15th December 2008, 19:22

PMSN sounds like Pre-recorded Media Serial Number defined in the AACS specs. It is read with the same command as the Volume ID, just the Format Code is different (0x81 instead of 0x80). And it has the same restrictions, you need to be AACS authenticated to get it. So without a new firmware patch we won't be able to get it currently :(.

:rolleyes:

loo3aem3ON

15th December 2008, 20:00

BDVM Debugger v0.1.5: download (http://uploaded.to/?id=xcco6l)

The new version of the debugger can now interact with DumpHD. So if everything works as expected all the BD+ damages are repaired in the background. :)

What yippiekayee reported is not a bug but more a cosmetical problem.
I create the file in case it is missing. Every stack dump indicates a bug ;)

loo3aem3ON

15th December 2008, 20:51

Thanks for your replies. I've tried to read the number with a patched drive (cmd AD handler is modified) and for format code 81h i get:
Fixed format, current; Sense key: Illegal Request
Additional sense: Cannot read medium - incompatible format

format code 82h:
Fixed format, current; Sense key: Illegal Request
Additional sense: Invalid field in cdb
Sense Key Specific: Error in Command byte 7

I can't tell if the results would be any different after successful authentication.

KenD00

15th December 2008, 23:33

I remember that i had the same results when i tried it with BD-ROM's and HD-DVD's, however i got a (somehow wrong looking) MediaID when using a non-AACS protected BD-RE. The MediaID should be present on recordables while the Volume ID is not.

However, i can't reproduce this behavior right know and i don't know why, the only thing i changed is that i have the patched YL05 firmware and that this time i misused aacskeys instead of DumpVID :confused:. Now i always get sense code 5/6F/00 - Authentication Failure.

I have also tested it now using my good old MKBv1 HD-DVD drive with aacs authentication and in that case (and using the XBox hack) i got in both cases 5/6F/01 - Key not present.

:rolleyes:

Accident

16th December 2008, 01:36

ID4:

[segment] Saving table 0 tableID 00000001, numSegments 408
[segment] Saving table 1 tableID 00000001, numSegments 408

Seems to be a bug in my conv_tab merger. They have the same IDs and should either be ignored, or, properly merged. But having the same number of Segments makes me think it is an identical table.

Edit:

if (set2->Tables[ ctable ].tableID ==
- set1->Tables[ i ].tableID) continue;
+ set1->Tables[ i ].tableID) break;

:(

Edit 2:

Looking at Benders Game, it seems to generate a good looking conv_tab, but it uses a lot of type 0, and type 3, repair descriptors. Used in such a way which makes me think we are not doing the correct logic, for example an entire Segment are all removed and left empty as all repair descriptors are either type 0 or type 3, and we assume both are identical to type 2.

Emp3r0r

16th December 2008, 06:39

And as far as holding back and letting Slysoft take the lead - that just backfired (http://it.slashdot.org/article.pl?sid=08/12/13/137257). This will make handling of the latest batch of BD+ titles a much bigger story than it ever could've been if they'd be handled by now. It is generally acknowledged that it might take some time to catch up with copy protection..

One potential flaw I just noticed in the way BD+ uses RSA is that they use the public exponent e = 3. This low value is known to open up multiple theoretical attacks as described in section 4 of this paper (http://crypto.stanford.edu/~dabo/abstracts/RSAattack-survey.html) [stanford.edu]. Too lazy to register a Doom9 account to post that info on their forums...
http://it.slashdot.org/comments.pl?sid=1061451&cid=26105869

loo3aem3ON

16th December 2008, 14:41

Looking at Benders Game, it seems to generate a good looking conv_tab, but it uses a lot of type 0, and type 3, repair descriptors. Used in such a way which makes me think we are not doing the correct logic, for example an entire Segment are all removed and left empty as all repair descriptors are either type 0 or type 3, and we assume both are identical to type 2.
Send me the content code please. Furthermore output the first 64 bytes on each call of TRAP_Finished and study the output for several movies. You should see some constant values (similar to C0200002 6420ED34 58010000 B801FFFC). There is also a table at 58h(?) which should look like this:
00 00 00 00 00 00 00 00 07 D6 06 0D 57 30 5D 64
D1 01 3F FF A4 D7 23 55 B4 C7 0B 27 A8 15 A3 43
05 E2 2B 42 F8 8A 5C 1A C5 E7 0D F4 66 35 3B DB
0A 83 22 8A 35 D2 90 CF 7E 2F 60 C1 AC 39 CE 1A
6D 31 08 CE F5 2F 91 C1 8E 39 1F 55 39 10 25 EA
DE 5A AE 40 92 D2 C0 6B 7E C2 2B DA 01 E7 B6 3E
83 95 E4 7B 1E F0 8A 2B AB F4 CB 6F D3 9D CC 71
F3 7A CF F7 23 07 32 69 9C D4 D5 07 E2 FC 4D 2B
59 21 A9 9D A4 7E EC DA E4 FF 76 7B 21 77 EF 20
2B 9A 2A BA FC CE 9D 1D 34 03 6D 36 BA 20 FB EE
21 93 A0 49 48 BC 13 82 14 04 B2 16 24 5B CD A4
FF DF 7D 2E EE FB A4 5D 43 67 55 80 E0 0D 57 BA
99 55 57 74 57 3C 00 AE 75 F2 13 8A 0B 38 5F D7
E4 C2 17 94 00 00 00 00 00 00 00 00 00 00 00 00
I don't know the meaning of these values but they might be useful to check if the segment keys/masks are correct. Keep in mind that the content code overwrites this area after TRAP_Finished with random data and it doesn't clear up the mess when writing new valid data and calling TRAP_Finished again.

Edit: Never mind. I found the content code for Benders Games.

Accident

17th December 2008, 01:35

Do you think it is more likely our key/mask is wrong, generating type 0, and 3. Or is it possible that it does in fact have a type 0/3 ?

With Benders Game, the masks we receive "seem" typical (most bits are set to 1), but the 5th mask "6E3158107F072F17", looks out of place and this is where type 0/3 starts. But that is hardly science.

Looking at the bytes after trap_Finish as you suggested, there is definitely a pattern (ID4):

Q:FAFEFBDF9AFF6EFFC02000026420272458010000B801FFFC40180CD2780D883F00000000000000
Q:FF3FDFFFF7DFFFDBC02000026420272458010000B801FFFC5B626DF440180CD200000000000000
Q:6BEDDDFF6767FE7BC02000026420272458010000B801FFFCFFFFFFFFFFFFFFFD00000005FFFFFF
Q:FF5C7F76FF6CDDEFC02000026420272458010000B801FFFCAE49044D7FFC262D00000005FFFFFF
Q:FEFFFFBFFF7DF77FC02000026420272458010000B801FFFC96D3A639B57F047B00000005FFFFFF
Q:FFFFF923FFFDDFEFC02000026420272458010000B801FFFC000000000000000000000000000000
Q:99FF4FDFDFFFCFFFC02000026420272458010000B801FFFCE67B5C9CF90AE60C00000000000000

The first 8 bytes is the mask for type 2. Then I always get a static string of "C02000026420272458010000B801FFFC" for ID4, then there is a 2nd lot of 8 bytes, similar to the mask. Different releases seems to have a different static string, but it remains constant within the release.

loo3aem3ON

17th December 2008, 12:39

I must say i am surprised none of the keys we use have been revoked (otherwise we wouldn't get a conversion table at all).

Do you think it is more likely our key/mask is wrong, generating type 0, and 3. Or is it possible that it does in fact have a type 0/3 ?
There is no type 0 or 3. If you open the conversion table in ConvTableView you will see many errors so clearly the segment keys are wrong. This should be easy to fix once i provide a new snapshot package. Maybe TRAP_DiscoveryRAM is the troublemaker :devil:

Looking at the bytes after trap_Finish as you suggested, there is definitely a pattern (ID4):
The pattern appears after all events on different addresses depending on the event id. Have you seen the table at 58h ?

Different releases seems to have a different static string, but it remains constant within the release.
Yes it seems it's neither status code nor checksum. :(

Could you and Rupan start implementing AACS in libblueray? :)

Accident

17th December 2008, 13:00

There is no type 0 or 3. If you open the conversion table in ConvTableView you will see many errors so clearly the segment keys are wrong. This should be easy to fix once i provide a new snapshot package. Maybe TRAP_DiscoveryRAM is the troublemaker :devil:

There is no type 0/3 YET :)

Yes, DiscoveryRAM is indeed just about the only trap before it goes haywire.

The pattern appears after all events on different addresses depending on the event id. Have you seen the table at 58h ?

No did not look that far, I will do.

Could you and Rupan start implementing AACS in libblueray? :)

I would love to. But I would need a lot of information to know what to do. I can only hope they are as good as your documentation, which has been perfect.

loo3aem3ON

17th December 2008, 13:26

There is no type 0/3 YET :)
I strongly believe that the BD+ specification doesn't introduce new features over time. My player ignores descriptors of type other than 1 or 2 and i see no reason to believe the implementation is incomplete. Isn't it more likely that you just have to flip one of the two most significant bits of the segment key to transform your type 0/3 descriptors to valid type 1/2 descriptors?

I would love to. But I would need a lot of information to know what to do.
The AACS specification is available here (http://www.aacsla.com/specifications/). If you have any questions KenD00 will probably be able to help you. For new keys i could ask the almighty Oopho2ei.

Accident

17th December 2008, 13:35

I strongly believe that the BD+ specification doesn't introduce new features over time. My player ignores descriptors of type other than 1 or 2

Ah, I wasn't sure if you had checked the refence player already or not, but with that knowledge we know that something is wrong if we see type 0/3.

The AACS specification is available here (http://www.aacsla.com/specifications/). If you have any questions KenD00 will probably be able to help you. For new keys i could ask the almighty Oopho2ei.

Some general info would also be nice. Is it at all like CSS? I query the device for a challenge, do some work to eventually be authenticated. After that, I do similar work to gain title keys, after that it is plain decrypting.

What is the deal with player firmware? We can only work with certain drives? Ie, is it not going to feasible to be a full BD player that needs no modification (beyond perhaps player keys). Should this be in a new thread?

loric

17th December 2008, 14:00

Some general info would also be nice. Is it at all like CSS? I query the device for a challenge, do some work to eventually be authenticated. After that, I do similar work to gain title keys, after that it is plain decrypting.

http://freedom-to-tinker.com/blog/felten/aacs-decryption-code-released

http://forum.doom9.org/showthread.php?t=122363

loo3aem3ON

17th December 2008, 14:18

I've started a new thread here (https://forum.doom9.org/showthread.php?t=143547). Please stop making AACS related postings in this thread.

Thank you.

Turtleggjp

29th December 2008, 19:10

Looks like Slysoft has finished cracking the latest BD+.

http://forum.slysoft.com/showthread.php?t=24603

I'm curious to see what you guys find out about this new protection and why it took so much longer for them to fix.

setarip_old

29th December 2008, 19:18

@Turtleggjp

Hi!Looks like Slysoft has finished cracking the latest BD+.That doesn't seem to be the case, if you read that entire thread...

pihug12

29th December 2008, 22:15

Press release : http://forum.slysoft.com/showthread.php?t=24602
BD+ Titles AnyDVD 6.5.0.2 may not handle correctly : http://forum.slysoft.com/showthread.php?t=24613

ron spencer

29th December 2008, 23:28

they are adding titles manually...see their forum

6.5.0.3 is now out

loric

29th December 2008, 23:59

they are adding titles manually...see their forum

6.5.0.3 is now out

But they're on the right track though. I am a Linux user but I might purchase their program. I think they deserve it.

loo3aem3ON

30th December 2008, 00:08

Looks like Slysoft has finished cracking the latest BD+.
Thank you. I'll get one of the new releases and record snapshots.

Accident

30th December 2008, 00:10

Woohoo, finally we can move on! It was good to have a break though :)

loo3aem3ON

31st December 2008, 00:50

A new snapshots package is available to developers now (write me a private message if you need it). It has been taken from a player with a more recent firmware which sadly also leaks more environment information through an extended TRAP_DiscoveryRAM (this is why the snapshot package is currently not public). The old firmware seems no longer be compatible and a picture with update instructions is displayed on the screen. The picture probably originates from the content code because it's different compared with earlier discs. It usually shows up when i mess up the input or return of TRAP_Aes/ TRAP_Privatekey.
Anyway the new version key (aes key 6) is not known yet and the new obfuscation scheme is more sophisticated so be patient.

loric

31st December 2008, 14:28

A new snapshots package is available to developers now (write me a private message if you need it). It has been taken from a player with a more recent firmware which sadly also leaks more environment information through an extended TRAP_DiscoveryRAM (this is why the snapshot package is currently not public). The old firmware seems no longer be compatible and a picture with update instructions is displayed on the screen. The picture probably originates from the content code because it's different compared with earlier discs. It usually shows up when i mess up the input or return of TRAP_Aes/ TRAP_Privatekey.
Anyway the new version key (aes key 6) is not known yet and the new obfuscation scheme is more sophisticated so be patient.

Judging from what you can see, what are the differences between the previous and the latest BD+ implementation (except for the different keys and the new obfuscation, which you have already told us about)?

loo3aem3ON

31st December 2008, 16:39

Judging from what you can see, what are the differences between the previous and the latest BD+ implementation (except for the different keys and the new obfuscation, which you have already told us about)?
TRAP_DiscoveryRAM has an extended range so the content code can access much more environmental data. Accident might have found more differences. I am working on aes key 6.

Accident: I noticed that the content code calls TRAP_DeviceAccess a lot more often and this trap returns data of various length. Looks like i have to take a look at the implementation of this system call as well.

Accident

1st January 2009, 02:55

Accident: I noticed that the content code calls TRAP_DeviceAccess a lot more often and this trap returns data of various length. Looks like i have to take a look at the implementation of this system call as well.

New player key 2, new aes key 6, and changes in DiscoveryRAM, including minor problems specific to libbluray. We've already discussed some of these in private.

I found this one particularly annoying:

[dlx] DiscoveryRAM(*ffffffff, *003DE79B, 00000001)
R1 : 80000001 == 00000000 : *NG*

Tripped by the (src + len < src) test. Test works best if 1) dst is bounds tested, 2) !len test, 3) src is bounds checked len-1.

That's all fine, but I'm more unsure about this situation:

[dlx] 0002A804 (I=39): TRAP([520]): TRAP 1312
[dlx] setWD (0x97000, 00000AB7 -> 7FFFFFFF)
[dlx] ** #004958 TRAP 00000520 (1312) return PC=0002A804
[dlx] trap_Discovery: 00000001, 00000003, 000BC434, 60
[dlx] trace_WD FAILED. WD=7FFFFFFF, trace-WD=00000977 (PC=0002A808)

I thought all TRAPs would reset WD, and yet the snapshot WD has a much lower value. This is before any breaks have occurred (trap #4958), but after trap_Finished. Does trap_Finished perhaps also modify the WD reset value?

Edit:

I stuck in this code for now:

dlx_setWD(vm, 0x7FFFFFFF);
dlx_trap(vm, U_Jimm);
vm->num_traps++;
if (vm->num_traps == 4959)
dlx_setWD(vm, 0x977);

And it carries on execution until it starts to post events. But it is the first trap after trap_Finished. dlx_trap() sets it to 4000 if it was finished() (which it did the previous call). It is perhaps something I do wrong though, so for now I'm not looking at it too deeply.

Edit2:

I did consider the possibility of not reseting WD at all, but the value of WD before calling the offending trap is 00000AB7. Much closer to 977 admittedly, but still I can not explain it.

loo3aem3ON

1st January 2009, 13:19

I found this one particularly annoying:

[dlx] DiscoveryRAM(*ffffffff, *003DE79B, 00000001)
R1 : 80000001 == 00000000 : *NG*

Tripped by the (src + len < src) test. Test works best if 1) dst is bounds tested, 2) !len test, 3) src is bounds checked len-1.
I've updated the parameter check for TRAP_DiscoveryRAM in the repository.

I thought all TRAPs would reset WD, and yet the snapshot WD has a much lower value. This is before any breaks have occurred (trap #4958), but after trap_Finished. Does trap_Finished perhaps also modify the WD reset value?
The timer makes sure the execution of the content code is interrupted every 4000 cycles to check for new events (you know). So if the content code doesn't process an event/callback the timer is _always_ reset to a specific reasonably low value. I am wondering why it's not reset to 0xFA0 in this case though.

loo3aem3ON

1st January 2009, 14:32

I stuck in this code for now:

dlx_setWD(vm, 0x7FFFFFFF);
dlx_trap(vm, U_Jimm);
vm->num_traps++;
if (vm->num_traps == 4959)
dlx_setWD(vm, 0x977);

And it carries on execution until it starts to post events. But it is the first trap after trap_Finished. dlx_trap() sets it to 4000 if it was finished() (which it did the previous call). It is perhaps something I do wrong though, so for now I'm not looking at it too deeply.
On a second thought it would make sense if the timer is not reset with a trap call (other than TRAP_Finished) if the content code is idle (not processing event/callback). Otherwise the content code could prevent any further event checks by calling arbitrary traps in an infinite loop.
I suggest you issue EVENT_Startup (event#0000) to start the content code. Furthermore use a flag which is set every time the player(libbluray) issues an event and cleared every time TRAP_Finished is called. Then for every trap call only reset the timer if that flag is set.

Edit:
I did consider the possibility of not reseting WD at all, but the value of WD before calling the offending trap is 00000AB7. Much closer to 977 admittedly, but still I can not explain it.
Then maybe this trap call takes about 0x140 clock cycles?

loo3aem3ON

3rd January 2009, 22:21

I'm making progress with aes key 6. How is it going for you? Could you fix many bugs and do you need more information? Remember not to post the output of TRAP_DiscoveryRAM please. ;)
Btw it seems the address ranges of this virtual player memory (TRAP_DiscoveryRAM) are vendor specific and each vendor has to provide the license administration with a memory footprint. This could mean each time the certificate 1 (player key 1 and aes key 6) is updated the implementation of TRAP_DiscoveryRAM is updated as well.

Accident

4th January 2009, 01:01

There have been a few minor fixes, and some confusing ones like the WD changes. I also get this situation:

[dlx] Aes(*003DFD48, *000A003C, 00000001, *000A002C, 00000006
[dlx] TRAP return: R1 = 80FFFFFF
R1 : 80FFFFFF == 80FFFFFF : ok
R28 : 000AD8F8 == 000AD904 : *NG*

So, for some reason it returns "INTERNAL_ERROR", which I guess is fine and I match that error. But as to why R28 is modified I do not know. Happens about 3 times in this snapshot run, with different values for R28. But it probably does not matter in the long run.

Edit:

Ok, it seems it is me doing it at the previous Break, when I restore WD and R28=PC:

[dlx] BREAK! PC=0005605C. WD=FFFFFFFF (old R28 000AD904)
[dlx] getPC (0x9a000): 0005605C
[dlx] setWD (0x9a000, FFFFFFFF -> 00000FA0)
snip
before R28 0005605C
snip
R28 : 0005605C == 000AD904 : *NG*

So, clearly I should not set WD "sometimes" (possibly even consider Traps to decrement WD if we are outside of event processing.) And "sometimes" not save R28. Hopefully those are the same "sometimes". This appears to be a libbluray problem, and I will work on it.

loo3aem3ON

4th January 2009, 14:00

So, for some reason it returns "INTERNAL_ERROR", which I guess is fine and I match that error.
I'll check why this happens.

But as to why R28 is modified I do not know. Happens about 3 times in this snapshot run, with different values for R28. But it probably does not matter in the long run.
Is it possible that r28 is already wrong before the trap call?

js06078

5th January 2009, 01:01

I took a look at the new snapshot-package, and here's a complete list of the problems that I found:

Conversion Table:

This call of TRAP_GetConversionTable gives as expected the (encrypted) conversion table:
#010166 TRAP_GetConversionTable(000F7D28, 002DFA68);

The conversion-table that is created by the debugger when using the snapshots (the snapshots are
decrypting the first ~0x30 segments of the sub-table with id="4") has a reasonable
structure (with all the 232 non-empty segments residing in sub-table with id="4") but all the non-empty
segments contain only 0x00s. This is probably the case since the current version of the debugger
filters out repair-descriptors of types 0 and 3.

It was mentioned earlier in this thread, that previous players ignored these types of repair-descriptors,
but in order to play the newest batch of BD+-titles, these players had to be updated. My guess is, that repair-
descriptors of type 0 and 3 are possible now.

Here is a list of types of repair descriptors:
0 - watermark descriptor?
1 - normal repair descriptor
2 - ignore repair descriptor
3 - watermark descriptor?

Idle Code:

As we know, there are two kinds of VM-code breaks: event-inducing breaks and idle-breaks.

The event-inducing breaks store some data at VM-memory 0x00-0x28 so that the VM can decide which
event was induced. Further, the VM's PC and WD are reset to 0x1000 and 0x7FFFFFFF respectively,
after an event-inducing break has happened.

Whenever TRAP_Finished is reached after an event was induced (and succesfully treated), the VM is
going to idle-mode as soon as WD<=0. There is a specific part of idle code which is being executed
as long as only idle-breaks are happening. The idle code is being executed in portions of 0xFA0
VM-cycles.

The whole concept of event-inducing breaks and idle-breaks is described somewhere earlier in this thread.

The only new thing in this version of BD+-content-code is, that the idle code contains trap-calls. This
has never been the case before.

The idle-code in the new snapshots-package is executing TRAP_AES with Key-ID = 6 and
TRAP_DeviceDiscovery(...,3,...) afterwards. Each trap seems to

have a fixed number of cycles assigned to it, indicating the time that it takes to execute this trap, or
the host system is executing the trap, counting the number of cycles <len> that it takes to execute the
trap and then sets WD to WD - <len>.

Anyway, all of this may be ignored, since we currently do not execute the idle-code at all (unless
snapshots are being used).

However, _if_ the correct execution of the normal content code should depend on the correct execution of
the traps in the idle-code, it may be necessary to introduce (a small number of) idle-breaks between
any two events.

The ECDSA-Keys seem to still be correct.

However, the parameter checking of TRAP_PrivateKey has a bug (this problem already appeared in previous
BD+-titles but was never fixed):
#010197 TRAP_PrivateKey (00000002, 003DF924, 03DF94C, 00000064, 80000000);
[!] register difference after trap call: r01: 80000001 (vm) != 80FFFFFF (snapshot)

TRAP_DeviceDiscovery has a new certificate for Question-ID = 2:
#000004 TRAP_DeviceDiscovery (00000001, 00000002, 00002CF8, 003DFF3C);
[!] memory difference after trap call: 002D0B: 02 (vm) != 04 (snapshot)
[!] memory difference after trap call: 002D0F: 03 (vm) != 01 (snapshot)
[!] memory difference after trap call: 002D54: 57 (vm) != 2D (snapshot)
[!] memory difference after trap call: 002D55: 30 (vm) != 1E (snapshot)
...
[!] memory difference after trap call: 002E18: E4 (vm) != 19 (snapshot)
[!] memory difference after trap call: 002E19: C2 (vm) != F2 (snapshot)
[!] memory difference after trap call: 002E1A: 17 (vm) != 04 (snapshot)
[!] memory difference after trap call: 002E1B: 94 (vm) != AE (snapshot)

This certificate is static and should hence be easy to adopt.

As already mentioned, TRAP_Aes has a new key for Key-ID = 6:
#000009 TRAP_Aes (0001CB88, 0001CB88, 00000002, 0001BBC8, 00000006);
[!] memory difference after trap call: 01CB88: 3E (vm) != DF (snapshot)
[!] memory difference after trap call: 01CB89: F7 (vm) != C9 (snapshot)
[!] memory difference after trap call: 01CB8A: 62 (vm) != E4 (snapshot)
[!] memory difference after trap call: 01CB8B: 0E (vm) != 57 (snapshot)
...
[!] memory difference after trap call: 01CBA4: C9 (vm) != 00 (snapshot)
[!] memory difference after trap call: 01CBA5: B2 (vm) != 00 (snapshot)
[!] memory difference after trap call: 01CBA6: 41 (vm) != 00 (snapshot)
[!] memory difference after trap call: 01CBA7: 65 (vm) != 05 (snapshot)

As soon as the new key is obtained, this problem should be fixed.

Here is another bug in the treatment of TRAP_Aes (this problem already appeared in previous
BD+-titles but was never fixed):
#010179 TRAP_Aes (003DE168, 003DE178, 00000000, 003DE118, FFF10000);
[!] register difference after trap call: r01: 00000000 (vm) != 80FFFFFF (snapshot)

As also already mentioned, TRAP_DiscoveryRAM is not handled correctly. This results in
a lot of errors. Here are some more small bugs in the parameter-checking:

#000469 TRAP_DiscoveryRAM (80000000, 003DEF98, 00000004);
[!] register difference after trap call: r01: 80000001 (vm) != 00000000 (snapshot)

#003892 TRAP_DiscoveryRAM (80200000, 003DFBDF, 00000001);
[!] register difference after trap call: r01: 80000001 (vm) != 00000000 (snapshot)
[!] memory difference after trap call: 3DFBDF: 18 (vm) != ** (snapshot)

The treatment of TRAP_Sha is still not correct for intermediate results. Anyway, this is not
a problem yet, since the content code does not check the intermediate results (yet).
This problem already appeared in previous BD+-titles but was never fixed:

#003456 TRAP_Sha (003DEA94, 003DEC94, 00000001, 00000001);
[!] memory difference after trap call: 3DEAA8: 00 (vm) != 61 (snapshot)

#003459 TRAP_Sha (003DEA94, 003DEC94, 00000001, 00000001);
[!] memory difference after trap call: 3DEAA8: 00 (vm) != 61 (snapshot)

#003464 TRAP_Sha (003DEA94, 003DEC97, 0000000F, 00000001);
[!] memory difference after trap call: 3DEAA8: 00 (vm) != 64 (snapshot)
[!] memory difference after trap call: 3DEAA9: 00 (vm) != 63 (snapshot)
[!] memory difference after trap call: 3DEAAA: 00 (vm) != 62 (snapshot)
[!] memory difference after trap call: 3DEAAB: 00 (vm) != 61 (snapshot)
...
[!] memory difference after trap call: 3DEAB3: 00 (vm) != 69 (snapshot)
[!] memory difference after trap call: 3DEAB4: 00 (vm) != 6F (snapshot)
[!] memory difference after trap call: 3DEAB5: 00 (vm) != 6E (snapshot)
[!] memory difference after trap call: 3DEAB6: 00 (vm) != 6D (snapshot)

TRAP_DeviceAccess is called several times.

Older BD+ title:
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 0F 00 FF FF

Newest snapshot-package:
TRAP_DeviceAccess (1,0): 00 00 01 00 -> F1 00 00 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> F1 00 00 00
TRAP_DeviceAccess (2,0)
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 00 40 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 00 40 00
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 00 80 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 00 80 00
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 00 C0 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 00 C0 00
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 01 00 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 01 00 00
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 01 40 00
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 01 40 00
TRAP_DeviceAccess (2,0): 00 00 00 00 -> 60 FB C4 BB
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 04 01 40 04
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 04 01 40 04
TRAP_DeviceAccess (2,0): 00 00 00 00 -> BC 79 59 5F
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 04 01 40 08
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 04 01 40 08
TRAP_DeviceAccess (2,0): 00 00 00 00 -> 97 F0 56 43
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 04 01 40 0C
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 04 01 40 0C
TRAP_DeviceAccess (2,0): 00 00 00 00 -> 07 A0 AB F2
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 04 01 40 10
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 04 01 40 10
TRAP_DeviceAccess (2,0): 00 00 00 00 -> CC 10 F7 C5
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 01 80 10
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 01 80 10
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)
---
TRAP_DeviceAccess (1,0): 00 00 01 00 -> 00 01 A0 10
TRAP_DeviceAccess (1,0): 00 00 00 00 -> 00 01 A0 10
TRAP_DeviceAccess (2,0)
TRAP_DeviceAccess (2,1)
TRAP_DeviceAccess (0,1)

The memory at the adress of the third parameter of this trap seems to hold another parameter
(0x00 or 0x100 so far). This parameter will be overwritten by the trap.

Furthermore, the trap returns somehow systematically changing values if called with parameters
(1,0). This could be some seeking instruction, returning the current 'position' on the disc?

If called with parameters (2,0), the trap seems to somehow obain 4 bytes of data from the disc.

TRAP_DeviceAccess (0,1) seems to signal the end of the device accessing.

If my guesses are right, the content code obtains the following 20 bytes from different positions
on the disc:
60 FB C4 BB BC 79 59 5F 97 F0 56 43 07 A0 AB F2 CC 10 F7 C5

Edit:

OK, the system behind the seeking seems to be:

The first call of
TRAP_DeviceAccess (1,0,*100) gives the maximum number (?) of the seeking value (in this case 0xF1000000)

The real seeking then starts with <position> = 0 in 400bytes-seek mode:
TRAP_DeviceAccess (1,0,*100) <position> += 400
TRAP_DeviceAccess (1,0,*0) <position> += 0
TRAP_DeviceAccess (2,0,*0) tries to obtain 4 bytes from <position>, and if successfull, then
TRAP_DeviceAccess (2,1,*0) sets to 4bytes-seek mode.

In 4bytes-seek mode:
TRAP_DeviceAccess (1,0,*100) <position> += 4
TRAP_DeviceAccess (1,0,*0) <position> += 0
TRAP_DeviceAccess (2,0,*0) tries to obtain 4 bytes from <position> and if finished reading, then
TRAP_DeviceAccess (2,1,*0) sets to 400bytes-seek mode again.

if in 4-bytes seek-mode, first byte of seek-result = 0x04,
if in 400-bytes seek-mode, first byte of seek-result = 0x00.

So basically, this goes through the disc, until it finds some readable data for the first time (in this
movie at 0x014000) and then reads some bytes from there in 4 byte steps (5 steps for this movie).
Probably these bytes will be compared with some fixed bytes from the content code.

The following problem appears in the treatment of every EVENT_getSegmentKey:
#010577 TRAP-SlotRead (003DF8C8, 00000000);
[!] memory difference after trap call: 3DF912: 01 (vm) != 00 (snapshot)
[!] memory difference after trap call: 3DF913: 13 (vm) != 65 (snapshot)

This should be easy to fix (if it is even necessary to fix it).

So to sum up my observations, here's a list of the important changes in this version of BD+-content code:

- Repair descriptors of type 0 and 3 appear for the first time.
- TRAP_DeviceAccess is used to obtain bytes from the disk for the first time.
- Player has new memory footprint (checked for by TRAP_DiscoveryRAM).
- Player has new AES-Key 6.
- Idle code executes traps.
- Player has new certificate 2.

loo3aem3ON

5th January 2009, 01:45

The conversion-table that is created by the debugger when using the snapshots (the snapshots are
decrypting the first ~0x30 segments of the sub-table with id="4") has a reasonable
structure (with all the 232 non-empty segments residing in sub-table with id="4") but all the non-empty
segments contain only 0x00s.
I don't think the segment keys get applied when running on snapshots so you are looking at an encrypted conversion table.

It was mentioned earlier in this thread, that previous players ignored these types of repair-descriptors,
but in order to play the newest batch of BD+-titles, these players had to be updated. My guess is, that repair-
descriptors of type 0 and 3 are possible now.
Accident was speculating about the existence of type 0 and 3 descriptors and so far there has been no evidence for that theory. I had to update my player because the AACS (not BD+!) keys got revoked (the disc the snapshots were taken from is MKBv9) so the content code was never executed. If you run this new content code with the old keys in the debugger you will get a conversion table which you surely wouldn't get if the keys/certificate had not been accepted. It's still unknown if BD+ even has a key/certificate revocation mechanism.

Here is a list of types of repair descriptors:
It wouldn't make sense to introduce a new descriptor type to add watermarks to the movie. An attacker could easily filter them out by ignoring this type of descriptor.

Each trap seems to

have a fixed number of cycles assigned to it, indicating the time that it takes to execute this trap, or
the host system is executing the trap, counting the number of cycles <len> that it takes to execute the
trap and then sets WD to WD - <len>.
Could you please create a list of timer differences for each trap type? :)

The ECDSA-Keys seem to still be correct.
The new certificate 1 and the corresponding new private key are already in the source code of the debugger (commented out -> see constants.java).

The memory at the adress of the third parameter of this trap seems to hold another parameter
(0x00 or 0x100 so far). This parameter will be overwritten by the trap.
Interesting.

Furthermore, the trap returns somehow systematically changing values if called with parameters
(1,0). This could be some seeking instruction, returning the current 'position' on the disc?
I should be able to observe the communication between player and drive. Assuming it really is TRAP_DeviceAccess it could also try to change the settings of other devices (network interface, audio/video hardware, ...). My player doesn't have a network connection.

Thanks a lot for your comprehensive report. :thanks:

Edit:
A general note: I often enumerate starting at zero so the second certificate is certificate 1 and the 7th aes key is aes key 6. Maybe we could agree on one form. The certificate 2 doesn't exist if we start counting at 0. :rolleyes:

Accident

5th January 2009, 09:59

Sorry for the delay, many projects distracting me.

[bdtest] executed 37820891 instructions, 10169 traps and 434 breaks (1)
[dlx] saving convTab to 'conv_tab.bin'
[segment] Saving table 0 tableID 00000000, numSegments 1
[segment] Saving table 1 tableID 00000001, numSegments 1
[segment] Saving table 2 tableID 00000002, numSegments 1
[segment] Saving table 3 tableID 00000003, numSegments 1
[segment] Saving table 4 tableID 00000004, numSegments 233
snip

As it turned out, the WD changes are as follows.
1) Post EVENT_Start (0x00) when we start and set "processing" flag true.
2) in TRAP_Finished if ("processing") { PC = R28; WD = 0xFA0; // 4000 decimal } AND clear "processing" flag.
3) For all other traps, if ("processing") { WD = 0x7fffffff; }
4) If ( NOT "processing" ) WD is left alone, or rather, decremented based on the duration of the trap execution, which is CPU based, and I can not see any way they could use this against us. Aes takes ~0x140.
5) WD is always set to 4000 on breaks (or it'd be 0 obviously).

The processing of R28 appears to work as follows:
1) In event posting (starting with EVENT_Start) { R28 = PC, PC=0x1000, WD=7fffffff, "processing" = 1 }
2) In trap_Finished, if ("processing") { PC=R28; WD=4000; } as above. Set "processing" = 0.
3) WD<= 0, Signal break, WD = 4000 and if ("processing") { R28 = PC; }

To sum up, I had to add the "processing_events" flag, and clear it on Finished. Only update some WD, and R28 if this flag was set. I was also missing the assignment of R28 in dlx_postevent().

ps. TRAP_Aes returns INTERNAL_ERROR on traps: 4961,6737,8443,8445,8447,8452,8454,8459,8461. Could they be due to "processing" flag? .ds

pps. I always count from 0 normally, but I think I may have said key2 earlier.

loo3aem3ON

6th January 2009, 12:42

The new obfuscated AES implementation is a simulation of a NAND gate ("not ( A and B)") netlist. For every key there is a different netlist. You can think of it as a hardware implementation (FPGA/CPLD/ASIC...) where the key is hardcoded into the VHDL/Verilog description and then synthesized to receive the netlist. Note that using NAND gates is logic complete that is you can construct any other gate function (AND, OR, NOT, XOR, ...) using only NAND gates.

You can download the package here (http://uploaded.to/?id=c3xvx2). It contains the netlist for key 4 (FEBE6C124354A6D3E19E491A816CCD4D) and key 6 (unknown) together with readout points. The readout points are the signals which finally contain the decryption result. The sample source code is currently set to key 6 but if you want to try this with key 4 you have to switch both the netlist and the readout file.

If you would like to have the netlists and readout points for the other keys (0-3,5) which we already know just ask.

loo3aem3ON

8th January 2009, 22:34

I've now properly implemented TRAP_Sha. The intermediate values seem to be correct. Any idea what calling SHA_Finished directly after SHA_Block is supposed to do? Should i reset the SHA-1 context after SHA_Block?

For those who are anxious to know what's causing the problems with the new content code: it's probably TRAP_DeviceAccess. I'll study the player implementation now. Thanks to js06078 for the suggestions.

Edit:
As you can see from the snapshot archive the segment keys are:
F4 C5 64 D3 4B B9 DF 6C 69 B9 8C AE F5 CF BD 6E
11 5A ED 49 D5 BC 63 DE 2D CE A6 78 65 CA F5 14
13 5A 33 F3 F2 12 AA 8F 82 0C B5 DB 4C 7E 3D 65
E8 6F FB 32 D7 17 23 18 8C FE 6B 0F 07 6D D3 EA
17 F0 B8 F1 CC A1 AB 3F C2 A9 E8 E6 3D E5 8F C1
68 41 00 29 7B 44 AD 51 9A 9B C8 A9 BB 13 6C 98
32 1B F3 7B E7 29 36 FD 76 48 B7 30 10 CC F4 20
6F CD 4A E4 2C 62 EF 48 C3 CE 52 60 0C A2 F2 00
C6 B4 41 0E 36 60 02 F0 70 13 B0 2B DB C2 2D BA
...
The debugger is way off :o

Edit2:
When the content code is idle trap calls seem to take 0x140 clock cycles (virtual machine clock cycles). I can confirm this for TRAP_Aes and TRAP_DeviceDiscovery. The number of clock cycles is calculated by fetching some value "0x3C" from memory and multiplying it by 2. Afterwards a constant "0xC8" is added to the result.

loo3aem3ON

9th January 2009, 23:47

Any idea what calling SHA_Finished directly after SHA_Block is supposed to do? Should i reset the SHA-1 context after SHA_Block?
The context is stored in a player specific format at 0x160.

SHA_UPDATE = read context -> process -> write context
SHA_INIT = process -> write context
SHA_FINAL = read context -> process -> write hash
SHA_BLOCK = process -> write hash

After reading the context on SHA_FINAL and SHA_UPDATE i get the same segment keys on every run now. That's significant progress although these keys are still wrong. :)

The sha-1 context of this player is 0x160 byes long and all values are stored little endian (relative to the beginning of the context block). The format is given below:
0x000-0x013 = Intermediate Hash
0x014-0x053 = Message Block
0x154-0x157 = Message Block Index
0x158-0x15F = Length [bits]
For more details see the source code of the debugger (classes: sha1_ctx and sha1).

Edit:
It looks like the new content code hashes the same block twice and interleaved in small steps. So if you don't read the context from pDst before processing you are unable to resume properly when switching to the other context again.

Accident

10th January 2009, 00:45

Edit:
It looks like the new content code hashes the same block twice and interleaved in small steps. So if you don't read the context from pDst before processing you are unable to resume properly when switching to the other context again.

I was thinking that would be exactly what I would do, as I wrote it. I believe I put in an "if" statement to detect it as well, but it just got drowned out in the noise of debug output.

Edit:

Great news about the 6th key. Thank you and thank You. I have updated libbluray with it, and trying to understand the memory changes.

loo3aem3ON

10th January 2009, 02:42

I've randomized the return of TRAP_DeviceAccess and against our expectations the segment keys don't change so we can ignore this trap for now. This seems to leave only TRAP_DiscoveryRAM and indeed if i randomize certain sections of the virtual player memory the segment keys begin to change. I am currently trying to isolate the memory areas we need to focus on.

loo3aem3ON

12th January 2009, 13:36

One of our contributers has just broken the new obfuscation scheme. The new AES key 6 (version key) is: A69907F63DD37D41588397A3D992A7E7

:thanks:

The critical area in the virtual player memory is 0x00250000-0x0028FFFF. It's no longer filled with the free space pattern instead some other simple but dynamic (system time depended?) pattern. The segment keys change by the hour of the day so it's possible that the time stamp (from TRAP_DeviceDiscovery) is used in the creation of this new pattern.

Edit: Everything above 0x3FFFFF can be ignored. A great deal of the address space 0x000000-0x3FFFFF is checksummed except for some small areas with "random" content (e.g. 0x00290000 - 0x00290123).

Edit2:
Some details about the structure of the pattern:0000-0003: constant "084EB164" (magic value?)
0004-0007: function return value
0008-000B:
000C-000F: "00000040" unknown (some value from stack)
0010-0013:
0014-0017:
0018-001B: function pointer
001C-001F:
As far as i can see it's an array of 32-bit little endian dwords. Parts of the code are obfuscated which usually means it's something interesting they are trying to hide. :rolleyes:

loo3aem3ON

14th January 2009, 01:18

[dlx] Aes(*003DFD48, *000A003C, 00000001, *000A002C, 00000006
[dlx] TRAP return: R1 = 80FFFFFF
R1 : 80FFFFFF == 80FFFFFF : ok
R28 : 000AD8F8 == 000AD904 : *NG*

So, for some reason it returns "INTERNAL_ERROR", which I guess is fine and I match that error.
I've looked into the code why this fails. Even before the regular parameter validation there is check if the current event id is 0x0000, 0x0010 or 0x0110. If the current event (callback) id is different from these values execution aborts with error code 0x80FFFFFF (internal error/refused). So it seems TRAP_Aes is forbidden during processing of callbacks/events 0x0210 and 0x0220. Maybe because these callbacks (events) do device communication?

One more thing: The current event id is set at the same time the program counter is reset to 0x1000 (as expected) but usually before the next event/callback is issued the current event id is set to 0x88888888. I suspect this is (indirectly?) done by TRAP_Finished. I can try to prove this if necessary.

Accident

14th January 2009, 02:42

I suspected that TRAP_Aes was not allowed to be called on occasion, but I assumed it was the "event processing" flag, and not event-ID checks. Thanks for figuring that out.

On the other note, I do not see the "literal" value of 0x88888888, but the content code frequently call Random, Memmove and DeviceDiscovery to over-write the 0x00000000 address. In addition to that, "idle execution" during Breaks seems to use it as a randomiser-counter, and a new value is observed every Break.

Edit:

If I did the test without using "event_processing" flag it fails around trap ~#3700, so I suspect a more-correct test is:

if ((eventid != 0x0000) &&
(eventid != 0x0010) &&
(eventid != 0x0110) &&
!vm->event_processing) {
debugf("[dlx] TRAP_Aes refused due to event-ID %08X\n", eventid);
result = STATUS_INTERNAL_ERROR;
}

This makes the snapshot testing pass all the TRAP_Aes "failed calls", all the way beyond trap #10178.

Edit 2:
Same rule applies for TRAP_PrivateKey. Quite possibly all traps in that grouping?

Edit 3:
By "current event id" i mean a special variable in the player (like in the decoder class of the debugger) and not the first dword of the vm memory.

Ah yes, so they chose to do the flag that way, and use 0x88888888 as a "not in event" mode. Then I understand, and will further modify my changes to check my "event-id" instead of VM[0x00000000].

loo3aem3ON

14th January 2009, 12:16

On the other note, I do not see the "literal" value of 0x88888888, but the content code frequently call Random, Memmove and DeviceDiscovery to over-write the 0x00000000 address. In addition to that, "idle execution" during Breaks seems to use it as a randomiser-counter, and a new value is observed every Break.
By "current event id" i mean a special variable in the player (like in the decoder class of the debugger) and not the first dword of the vm memory.

If I did the test without using "event_processing" flag it fails around trap ~#3700, so I suspect a more-correct test is:
The debugger sets eventid to 0x88888888 on every call of TRAP_Finished now. It's just a constant in the player code and doesn't originate from the content code in any way. The player also never writes 0x88888888 to the event parameter area (0x00-0x3F). Maybe it's part of the BD+ specification or just bad programming to invalidate the current event id.

Your implementation is a better i think. :)

Same rule applies for TRAP_PrivateKey. Quite possibly all traps in that grouping?
I'll check the other traps in this group.

loo3aem3ON

14th January 2009, 13:20

Same rule applies for TRAP_PrivateKey. Quite possibly all traps in that grouping?
I went through all traps searching for other event id checks. This is what i found:
0x0110: if ( ceid != 0x0000 && ceid != 0x0010 && ceid != 0x0110 ) return 0x80FFFFFF;
0x0120: if ( ceid != 0x0000 && ceid != 0x0010 && ceid != 0x0110 ) return 0x80FFFFFF;
0x0510: if ( ceid == 0x0000 || ceid == 0x0010 || ceid == 0x0110 ) return 0x80FFFFFF;
0x0540: if ( ceid != 0x0000 && ceid != 0x0010 && ceid != 0x0110 ) return 0x80FFFFFF;
0x0550: if ( ceid != 0x0000 ) return 0x80FFFFFF;
Note: ceid = current event id which is 0x88888888 when content code is idle

Edit: the number before the colon is the trap id

Accident

15th January 2009, 03:27

Excellent, I have applied those tests as well. I finally also found a real problem in libbluray:

[dst] MemSearch(003BF9B0, 00000014, 003BF9C2, 00000002, 003BF8F0
[dlx] MemSearch: storing 00000000 in *dst
[bdtest] comparing with 'post_trap_snapshots/post_trap_reg_010911.bin'
+003BF8F0 00 00 00 00 E2 70 1A D3 00 00 01 00 00 00 02 00 |.....p..........|
-003BF8F0 00 3B F9 C2 E2 70 1A D3 00 00 01 00 00 00 02 00 |.....p..........|

Which was off-by-one error, probably only related to C version.

Edit: Not too surprising I suppose, but I had my trace-WD simply fix the differences when they occurred, and turning that off is enough for it to derail. So it does seem to depend on the 0x140 differences or others. Also amusingly enough, TRAPs (at #10147) will cause WD to trigger, which requires some special handling. It seems to just set it to 4000 without generating a BREAK though.

loo3aem3ON

15th January 2009, 12:46

We need to reverse engineer TRAP_DiscoveryRAM for memory range (0x00250000 - 0x0028FFFF) and (0x00290200 - 0x00290293). I'm taking snapshots of these sections after every trap call now. It is the only system call which still returns wrong data (for the given content code up to the point where the first segment key is returned).

Edit: The structure of the snapshots is:
00000000-00000003: last instruction (E400????)
00000004-0000001B: 6 parameters
0000001C-0000001F: trap return value
00000020-0004001F: snapshot (00250000-0028FFFF)
everything little endian

Edit:
Which was off-by-one error, probably only related to C version.
The debugger once had the same problem and this was fixed some time ago. :p

Not too surprising I suppose, but I had my trace-WD simply fix the differences when they occurred, and turning that off is enough for it to derail. So it does seem to depend on the 0x140 differences or others. Also amusingly enough, TRAPs (at #10147) will cause WD to trigger, which requires some special handling. It seems to just set it to 4000 without generating a BREAK though.
I'll give a short summary. Maybe you are making things to complicated.

If the content code is idle:
Trap calls cost 0x140 cycles if executed successfully. If the trap execution aborts with an error the timer remains unchanged. At the end of every instruction (this includes the TRAP instruction) the timer is decremented and subsequently checked if still greater than zero. If it's zero or below it is reset to 0xFA0.

If the content code is busy:
Basically identical to the above case except trap calls cost 0x7FFFFFFF cycles and the timer reset value is 0x7FFFFFFF.

TRAP_Finished and event set the timer to 0xFA0 or 0x7FFFFFFF and therefor require special treatment.

loo3aem3ON

15th January 2009, 23:01

A snapshot package of the virtual player memory range 00250000-0028FFFF is now available. The structure is as described in the previous posting. Each snapshot was taken after a trap call from the content code.
The snapshot data is BIG endian and the header little endian. Note that i already removed the obfuscation layer "^= ( 3 * address * address + 1 )".

Looking at the first 1000 snapshots i noticed that this area seems to change only after TRAP_Aes with player keys 3,4,6 so there might be a relationship. Also note that the player is caching the decrypted key so it won't decrypt the same encrypted key with the same player key again and again.

Send me a private message if you would like to help in the analysis of the contents of this section.

loo3aem3ON

16th January 2009, 22:25

Accidents recent findings show that TRAP_DeviceAccess is involved in the segment key calculation which i claimed is not. I've now double checked my previous results. It's easy to see that running the content code with snapshot guidance up to a certain point and then resuming without snapshot support can always be used to identify the last problematic trap call (using divide&conquer to search efficiently). I've now applied this technique to the new content code and the last problem is:
#005013 TRAP_DeviceAccess (00000002, 00000000, 003DF9E0);
My apologies. :devil:

Edit: the second parameter of TRAP_DeviceAccess seems to be the data direction ( 0 = write, 1 = read )
(0,1,?) : reads 4 bytes from pBuf (doesn't write)
(1,0,?) : writes 4 bytes to pBuf (doesn't read)
(2,1,?) : reads 4 bytes from pBuf (doesn't write)
(2,0,?) : writes 4 bytes to pBuf (doesn't read)

Accident

17th January 2009, 14:53

Not entirely sure what is wrong with libbluray's TRAP_PrivateKey, I updated key 1 (x, y and d) with the new values, but it still produces quite different results.

TRAP 000851

[dlx] PrivateKey(00000001, *00002CA8, *00002C94, 00000010, 00000000)
Computed message:
42 44 53 56 4D 5F 50 4B 00 00 00 00 00 00 00 10 31 96 92 A9 31 EB BC 05 0A B3 B8 3D 45 6F 36 01
| controlwd | srcLen | Data, 16 bytes. ->

SHA Hash computed from that message :
AD 03 B0 6A 8D 91 FF 63 21 8C 70 F1 68 7A 64 3B CB 15 B6 72

Computed signature, starting 8 bytes in. First 4 bytes are in the message above, so data copy is ok:
+00002CA0 45 6F 36 01 00 00 00 04 69 85 69 3B B8 58 9F 2E |Eo6.....i.i..X..|
+00002CB0 CD AD DF 56 95 0C FA 70 C0 8F 0F 4C 2D 11 8F CB |...V...p...L....|
+00002CC0 2E 76 0B 17 0C A8 BC 97 F4 47 B9 72 B8 D5 25 BA |.v.......G.r....|

Expected signature.
-00002CA0 45 6F 36 01 00 00 00 04 45 8C F0 B1 E0 26 05 E5 |Eo6.....E.......|
-00002CB0 1A F5 44 12 78 06 B7 1A 70 EA 43 32 71 0D D8 5B |..D.x...p.C2q...|
-00002CC0 9D 82 19 54 67 6F 6D F8 F8 F5 D6 BE 48 E8 BA D5 |...Tgom.....H...|

Perhaps I should spend some time to get the Java debugger to run so I can dump the interim values there and see where it goes wrong.

Edit:
Ah, this I was not aware of. That explains why the Debugger calls Verify after. Perhaps I should as well, to feel more secure.

loo3aem3ON

17th January 2009, 15:03

Not entirely sure what is wrong with libbluray's TRAP_PrivateKey, I updated key 1 (x, y and d) with the new values, but it still produces quite different results.
That's perfectly correct and the strength of ECDSA is based on this randomness. Lot's of signatures (r,s pairs) are valid for the same message/hash. The content code will verify the signature and will abort prematurely if the signature is wrong.

loo3aem3ON

17th January 2009, 21:42

I've traced the values returned by TRAP_DeviceAccess backwards and ended up in the java virtual machine interpreter :confused:
Is it possible that this trap is interacting with the java program (gui?) on the disc?
Could anyone give me a short summary of what the java program on the blue ray discs does? My current understanding is that it creates the animated gui for the user. :o

Edit: Btw the values returned by TRAP_DeviceAccess(2,0,?) seem to be different each time.

KenD00

17th January 2009, 23:28

My current understanding is that it creates the animated gui for the user.

Thats one function. However, it has some more advanced features too, e.g. it can access the AACS layer to provide online functions and managed copy. I don't have the BD+ specs (lol, who on this forum has them :D) but maybe the BD-J application can access the BD+ stuff too? But what sense would that make? BD+ protected online content? But this shouldn't affect the actual decryption process of the disc, except the BD-J code actually works together with the BD+ content code to decrypt the disc :confused:.

:rolleyes:

dirio49

17th January 2009, 23:33

Hi,
Searching though Google
it seems that TRAP_DeviceAccess has 4 parameters, not 3
UINT32 TRAP_DeviceAccess(UINT32 dev, UINT32 opID, UINT8*buf, UINT32*len);
http://www.freepatentsonline.com/y2008/0137848.html

Sorry if you already knew this

loo3aem3ON

18th January 2009, 00:09

But this shouldn't affect the actual decryption process of the disc, except the BD-J code actually works together with the BD+ content code to decrypt the disc :confused:
I've read somewhere that BD+ is sitting on top of BD-J so it could be that the data is only forwarded through this BD-J layer. I'm not sure about what i saw but it looks like the BD-J code is calling some function (callback?) which writes the 4 bytes into the memory. The data is later fetched by TRAP_DeviceAccess. The load/store procedures used to exchange the data look like the "shared memory" (shared between BD+ and BD-J) is well structured using 128 registers of length 32-bit each. The registers in use during TRAP_DeviceAccess activity are 67h and 68h.
(I haven't seen this code before and some of the statements above are based on assumptions.)

it seems that TRAP_DeviceAccess has 4 parameters, not 3
It's likely that the length was omitted because this trap seems to read or write a single 32-bit dword on each call only. I am not even sure it really is TRAP_DeviceAccess although the name seems to fit in alphabetically. But thanks for pointing out this difference.

loo3aem3ON

18th January 2009, 21:10

I've made a list of the important TRAP_DeviceAccess calls showing the exchange of data between the content code and some unknown device. A similar list has previously been made by js06078 in posting #608 (https://forum.doom9.org/showpost.php?p=1232293&postcount=608) but it didn't show the data transmitted from the content code to the device.

Send == from content code to device
Receive == from device to content code

#004963 TRAP_DeviceAccess(1, 0, *) // Receive "F1000000"
#004964 TRAP_DeviceAccess(1, 0, *) // Receive "F1000000"
#004965 TRAP_DeviceAccess(2, 0, *) // Receive "00000000"
#004966 TRAP_DeviceAccess(2, 1, *) // Send "00000001"
#004967 TRAP_DeviceAccess(2, 1, *) // Send "FFFFFFFE"
#004968 TRAP_DeviceAccess(0, 1, *) // Send "04000004"

#004970 TRAP_DeviceAccess(1, 0, *) // Receive "00004000"
#004971 TRAP_DeviceAccess(1, 0, *) // Receive "00004000"
#004972 TRAP_DeviceAccess(2, 1, *) // Send "3EA9DAFF"
#004973 TRAP_DeviceAccess(0, 1, *) // Send "04000008"

#004975 TRAP_DeviceAccess(1, 0, *) // Receive "00008000"
#004976 TRAP_DeviceAccess(1, 0, *) // Receive "00008000"
#004977 TRAP_DeviceAccess(2, 1, *) // Send "B23259D5"
#004978 TRAP_DeviceAccess(0, 1, *) // Send "0400000C"

#004980 TRAP_DeviceAccess(1, 0, *) // Receive "0000C000"
#004981 TRAP_DeviceAccess(1, 0, *) // Receive "0000C000"
#004982 TRAP_DeviceAccess(2, 1, *) // Send "53B07B58"
#004983 TRAP_DeviceAccess(0, 1, *) // Send "04000010"

#004985 TRAP_DeviceAccess(1, 0, *) // Receive "00010000"
#004986 TRAP_DeviceAccess(1, 0, *) // Receive "00010000"
#004987 TRAP_DeviceAccess(2, 1, *) // Send "60FBC4BB"
#004988 TRAP_DeviceAccess(0, 1, *) // Send "04000014"

#004990 TRAP_DeviceAccess(1, 0, *) // Receive "00014000"
#004991 TRAP_DeviceAccess(1, 0, *) // Receive "00014000"
#004992 TRAP_DeviceAccess(2, 0, *) // Receive "60FBC4BB"
#004993 TRAP_DeviceAccess(2, 1, *) // Send "60FBC4BC"
#004994 TRAP_DeviceAccess(0, 1, *) // Send "00000014"

#004996 TRAP_DeviceAccess(1, 0, *) // Receive "04014004"
#004997 TRAP_DeviceAccess(1, 0, *) // Receive "04014004"
#004998 TRAP_DeviceAccess(2, 0, *) // Receive "BC79595F"
#004999 TRAP_DeviceAccess(0, 1, *) // Send "00004014"

#005001 TRAP_DeviceAccess(1, 0, *) // Receive "04014008"
#005002 TRAP_DeviceAccess(1, 0, *) // Receive "04014008"
#005003 TRAP_DeviceAccess(2, 0, *) // Receive "97F05643"
#005004 TRAP_DeviceAccess(0, 1, *) // Send "00008014"

#005006 TRAP_DeviceAccess(1, 0, *) // Receive "0401400C"
#005007 TRAP_DeviceAccess(1, 0, *) // Receive "0401400C"
#005008 TRAP_DeviceAccess(2, 0, *) // Receive "07A0ABF2"
#005009 TRAP_DeviceAccess(0, 1, *) // Send "0000C014"

#005011 TRAP_DeviceAccess(1, 0, *) // Receive "04014010"
#005012 TRAP_DeviceAccess(1, 0, *) // Receive "04014010"
#005013 TRAP_DeviceAccess(2, 0, *) // Receive "CC10F7C5"
#005015 TRAP_DeviceAccess(2, 1, *) // Send "10000000"
#005016 TRAP_DeviceAccess(0, 1, *) // Send "04010018"

#005018 TRAP_DeviceAccess(1, 0, *) // Receive "00018010"
#005019 TRAP_DeviceAccess(1, 0, *) // Receive "00018010"
#005020 TRAP_DeviceAccess(2, 1, *) // Send "00000000"
#005021 TRAP_DeviceAccess(0, 1, *) // Send "0201001A"

#005023 TRAP_DeviceAccess(1, 0, *) // Receive "0001A010"
#005024 TRAP_DeviceAccess(1, 0, *) // Receive "0001A010"
#005025 TRAP_DeviceAccess(2, 0, *) // Receive "00000000"
#005026 TRAP_DeviceAccess(2, 1, *) // Send "00000001"
#005027 TRAP_DeviceAccess(0, 1, *) // Send "0001001A"

Your ideas about what is happening here and what kind of device/module the content code could be communicating with are welcome.

Edit: About 16 bytes are sent and then 16 totally different bytes are received. Maybe a cryptographic coprocessor? :confused:

tteich

19th January 2009, 15:31

I've made a list of the important TRAP_DeviceAccess calls showing the exchange of data between the content code and some unknown device. A similar list has previously been made by js06078 in posting #608 (https://forum.doom9.org/showpost.php?p=1232293&postcount=608) but it didn't show the data transmitted from the content code to the device.

Send == from content code to device
Receive == from device to content code

#004963 TRAP_DeviceAccess(1, 0, *) // Receive "F1000000"
#004964 TRAP_DeviceAccess(1, 0, *) // Receive "F1000000"
#004965 TRAP_DeviceAccess(2, 0, *) // Receive "00000000"
#004966 TRAP_DeviceAccess(2, 1, *) // Send "00000001"
#004967 TRAP_DeviceAccess(2, 1, *) // Send "FFFFFFFE"
#004968 TRAP_DeviceAccess(0, 1, *) // Send "04000004"

#004970 TRAP_DeviceAccess(1, 0, *) // Receive "00004000"
#004971 TRAP_DeviceAccess(1, 0, *) // Receive "00004000"
#004972 TRAP_DeviceAccess(2, 1, *) // Send "3EA9DAFF"
#004973 TRAP_DeviceAccess(0, 1, *) // Send "04000008"

#004975 TRAP_DeviceAccess(1, 0, *) // Receive "00008000"
#004976 TRAP_DeviceAccess(1, 0, *) // Receive "00008000"
#004977 TRAP_DeviceAccess(2, 1, *) // Send "B23259D5"
#004978 TRAP_DeviceAccess(0, 1, *) // Send "0400000C"

#004980 TRAP_DeviceAccess(1, 0, *) // Receive "0000C000"
#004981 TRAP_DeviceAccess(1, 0, *) // Receive "0000C000"
#004982 TRAP_DeviceAccess(2, 1, *) // Send "53B07B58"
#004983 TRAP_DeviceAccess(0, 1, *) // Send "04000010"

#004985 TRAP_DeviceAccess(1, 0, *) // Receive "00010000"
#004986 TRAP_DeviceAccess(1, 0, *) // Receive "00010000"
#004987 TRAP_DeviceAccess(2, 1, *) // Send "60FBC4BB"
#004988 TRAP_DeviceAccess(0, 1, *) // Send "04000014"

#004990 TRAP_DeviceAccess(1, 0, *) // Receive "00014000"
#004991 TRAP_DeviceAccess(1, 0, *) // Receive "00014000"
#004992 TRAP_DeviceAccess(2, 0, *) // Receive "60FBC4BB"
#004993 TRAP_DeviceAccess(2, 1, *) // Send "60FBC4BC"
#004994 TRAP_DeviceAccess(0, 1, *) // Send "00000014"

#004996 TRAP_DeviceAccess(1, 0, *) // Receive "04014004"
#004997 TRAP_DeviceAccess(1, 0, *) // Receive "04014004"
#004998 TRAP_DeviceAccess(2, 0, *) // Receive "BC79595F"
#004999 TRAP_DeviceAccess(0, 1, *) // Send "00004014"

#005001 TRAP_DeviceAccess(1, 0, *) // Receive "04014008"
#005002 TRAP_DeviceAccess(1, 0, *) // Receive "04014008"
#005003 TRAP_DeviceAccess(2, 0, *) // Receive "97F05643"
#005004 TRAP_DeviceAccess(0, 1, *) // Send "00008014"

#005006 TRAP_DeviceAccess(1, 0, *) // Receive "0401400C"
#005007 TRAP_DeviceAccess(1, 0, *) // Receive "0401400C"
#005008 TRAP_DeviceAccess(2, 0, *) // Receive "07A0ABF2"
#005009 TRAP_DeviceAccess(0, 1, *) // Send "0000C014"

#005011 TRAP_DeviceAccess(1, 0, *) // Receive "04014010"
#005012 TRAP_DeviceAccess(1, 0, *) // Receive "04014010"
#005013 TRAP_DeviceAccess(2, 0, *) // Receive "CC10F7C5"
#005015 TRAP_DeviceAccess(2, 1, *) // Send "10000000"
#005016 TRAP_DeviceAccess(0, 1, *) // Send "04010018"

#005018 TRAP_DeviceAccess(1, 0, *) // Receive "00018010"
#005019 TRAP_DeviceAccess(1, 0, *) // Receive "00018010"
#005020 TRAP_DeviceAccess(2, 1, *) // Send "00000000"
#005021 TRAP_DeviceAccess(0, 1, *) // Send "0201001A"

#005023 TRAP_DeviceAccess(1, 0, *) // Receive "0001A010"
#005024 TRAP_DeviceAccess(1, 0, *) // Receive "0001A010"
#005025 TRAP_DeviceAccess(2, 0, *) // Receive "00000000"
#005026 TRAP_DeviceAccess(2, 1, *) // Send "00000001"
#005027 TRAP_DeviceAccess(0, 1, *) // Send "0001001A"

Your ideas about what is happening here and what kind of device/module the content code could be communicating with are welcome.

Edit: About 16 bytes are sent and then 16 totally different bytes are received. Maybe a cryptographic coprocessor? :confused:
16bytes smells like DES, perhaps a CBC or ECB. But since AES supports different blocksizes it could also be a 16byte AES encryption / decryption. Some of the patents show an embedded security chip (kind of SmartCard chip). This could be one of the "devices" that the content code is able to access.

loo3aem3ON

19th January 2009, 19:04

16bytes smells like DES, perhaps a CBC or ECB. But since AES supports different blocksizes it could also be a 16byte AES encryption / decryption. Some of the patents show an embedded security chip (kind of SmartCard chip). This could be one of the "devices" that the content code is able to access.
DES uses a 56-bit key and not 128. I need more time to trace the data through the java virtual machine.

loo3aem3ON

20th January 2009, 15:29

One of our contributers gave us a list with the correct trap names. Below you can see those we named incorrectly:
0510 TRAP_ApplicationLayer (former TRAP_DeviceAccess)
0520 TRAP_Discovery (former TRAP_DeviceDiscovery)
0020 TRAP_FixUpTableSend (former TRAP_GetConversionTable)
0550 TRAP_MediaCheck (former TRAP_MediaSHAFileHash)
0220 TRAP_MultiplyWithCarry (former TRAP_MultiplyWithRipple)
0140 TRAP_Sha1 (former TRAP_Sha)
As you can see there is no TRAP_DeviceAccess and the correct name for this system call is TRAP_ApplicationLayer. This explains why i ended up in the java virtual machine interpreter with my trace.

Thanks a lot for this information! :thanks:

loo3aem3ON

20th January 2009, 21:34

To implement TRAP_ApplicationLayer allocate an array of 128 32-bit integer. The first parameter of TRAP_ApplicationLayer is the register number + 66h you are accessing. The second parameter specifies the direction (read/write or send/receive respectively).

Examples:
TRAP_ApplicationLayer( r, 1, 1234h ) == read the 32-bit integer at 1234h (vm memory) and write it to register 66h + r
TRAP_ApplicationLayer( r, 0, 1234h ) == read the register 66h + r and write its contents to 1234h (vm memory)

This array you are accessing is also present in the virtual player memory. If you call TRAP_DiscoveryRAM and request the range 0x00304000-003041FFF you will get it.

The folder BDMV/JAR/ on this disc contains a file 77770.jar. After extraction i could find the files BDClassLoaderException.class, BDPlusSocketException.class, bluray.Handshake.perm, Handshake.class and LibHandshakeException.class and others in the folder com/macrovision/bdplus/.

I can see the java opcode which is loading and storing the data in this array but i couldn't find it on the disc. The code contains lots of strings like areInputMethodsEnabled() so it is maybe a common java library.

Accident

21st January 2009, 03:41

I can confirm that it thinks TRAP_ApplicationLayer is 0x510, and takes 3 arguments.

I also noticed TRAP_FixUpTableSend() can return 0x80FFFFFF when "[$a0 + $9C] != 3". I am not sure what that check is, or what $a0 contains. Perhaps it is not important.

loo3aem3ON

21st January 2009, 17:29

I can confirm that it thinks TRAP_ApplicationLayer is 0x510, and takes 3 arguments.
Thanks.

I also noticed TRAP_FixUpTableSend() can return 0x80FFFFFF when "[$a0 + $9C] != 3". I am not sure what that check is, or what $a0 contains. Perhaps it is not important.
Yes i have ignored a few checks. I can see that TRAP_FixUpTableSend can return 80FFFFFFh too and it probably is important but other things like TRAP_ApplicationLayer have a higher priority now.

I've run a few tests with a BD+ protected disc without any java code. In this case TRAP_ApplicationLayer seems to be accessing a simple memory of 3 registers. Nothing else is changing the contents of these registers.

I have further tried other discs with different java code and they already behave differently when reading register 67h (calling TRAP_ApplicationLayer(1,0,*)). For example for "The Day After Tomorrow" i get F0000000h instead of F1000000h.

I was given a few debug messages which suggest that these registers are named PSR:

66h = PSR 102
67h = PSR 103
68h = PSR 104 (shared status register)

loo3aem3ON

22nd January 2009, 18:58

From the bits and pieces I was given i can tell that the registers we are looking at are called "Player Status Registers" (PSR) and for some of them i could find a short description (see this (https://trac.assembla.com/bdplus/browser/trunk/doc/misc/player_status_register.txt) file). A part of the data i received also suggests a relationship between PSR103 and AACS.

Edit: Renamed "Player State Registers" to "Player Status Registers" (thanks KenD00)

KenD00

22nd January 2009, 21:51

PSR sounds familiar, they are mentioned in the AACS spec but called Player Status Registers there, looks like they are the same. From what you have written in your documentation it looks like they are, in comparison to DVD-Players, the SPRM's of the Blu-Ray player. The AACS spec says that PSR96 and PSR97 contain the Playlist_Indicators when playing back a title that uses Sequence Keys.

:rolleyes:

loo3aem3ON

22nd January 2009, 23:02

PSR sounds familiar, they are mentioned in the AACS spec but called Player Status Registers there
I've just double checked my notes and indeed they are called "Player Status Registers". My mistake, sorry. At least the meaning is about the same. :rolleyes:

looks like they are the same. From what you have written in your documentation it looks like they are, in comparison to DVD-Players, the SPRM's of the Blu-Ray player. The AACS spec says that PSR96 and PSR97 contain the Playlist_Indicators when playing back a title that uses Sequence Keys.
Thank you. I've updated the documentation for PSR96 and PSR97. If anyone finds a description for some of the remaining unknown registers please post it here. It's required for a proper implementation of TRAP_DiscoveryRAM and also helps us with TRAP_ApplicationLayer.

loo3aem3ON

23rd January 2009, 10:29

I've updated the documentation of the player status registers again using various patents and this (http://forum.doom9.org/showthread.php?t=125903) posting by Pelican9.

I furthermore suggest to reverse engineer the java application as a short term solution. In the long run we need a fully functional BD-J interpreter + libraries.

loo3aem3ON

24th January 2009, 13:50

The logfile below shows in greater detail the communication between the java application and the content code:

// java code runs. It ...
// 1. writes PSR104 (set to 0x00000000)
// 2. writes PSR103 (set to 0xF1000000)
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004963 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0xF1000000)
#004964 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0xF1000000)
#004965 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0x00000000)
#004966 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x00000001)
#004967 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0xFFFFFFFE)
#004968 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x04000004) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004970 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00004000)
#004971 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00004000)
#004972 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x3EA9DAFF)
#004973 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x04000008) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004975 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00008000)
#004976 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00008000)
#004977 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0xB23259D5)
#004978 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x0400000C) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004980 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0000C000)
#004981 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0000C000)
#004982 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x53B07B58)
#004983 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x04000010) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004985 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00010000)
#004986 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00010000)
#004987 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x60FBC4BB)
#004988 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x04000014) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004990 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00014000)
#004991 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00014000)
#004992 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0x60FBC4BB)
#004993 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x60FBC4BC)
#004994 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x00000014) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR104
// 3. writes PSR104 again
// 4. writes PSR103
// 5. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#004996 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014004)
#004997 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014004)
#004998 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0xBC79595F)
#004999 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x00004014) and start execution of java code (see below)

// java code runs. It ...
// 1. writes PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#005001 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014008)
#005002 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014008)
#005003 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0x97F05643)
#005004 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x00008014) and start execution of java code (see below)

// java code runs. It ...
// 1. writes PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#005006 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0401400C)
#005007 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0401400C)
#005008 TRAP_DeviceAccess(2, 0, *) // read PSR104 (:= 0x07A0ABF2)
#005009 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x0000C014) and start execution of java code (see below)

// java code runs. It ...
// 1. writes PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#005011 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014010)
#005012 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x04014010)
#005013 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0xCC10F7C5) <-- last value the segment keys depend on
#005015 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x10000000) <-- java code runs shortly afterwards ?! (no read/write access to PSRs)
#005016 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x04010018) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#005018 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00018010)
#005019 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x00018010)
#005020 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x00000000)
#005021 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x0201001A) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR103
// 3. issues callback/event#0210 ?

// content code event#0210 handler called executing:
#005023 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0001A010)
#005024 TRAP_DeviceAccess(1, 0, *) // read PSR103 (== 0x0001A010)
#005025 TRAP_DeviceAccess(2, 0, *) // read PSR104 (== 0x00000000)
#005026 TRAP_DeviceAccess(2, 1, *) // write PSR104 (:= 0x00000001)
#005027 TRAP_DeviceAccess(0, 1, *) // write PSR102 (:= 0x0001001A) and start execution of java code (see below)

// java code runs. It ...
// 1. reads PSR104
// 2. writes PSR104
// issues next event/callback?

Things i noticed:
- TRAP_ApplicationLayer(0, 1, *) starts the execution of the java code
- other calls of TRAP_ApplicationLayer seem to access the Player State Registers only
- java code still runs when TRAP_Finished is invoked (or TRAP_Finished calls some java functions as well?)
- it could be that some of the events are caused by the java application (see how the content code and the java code are active interchangeably -> synchronization?)
- some java opcodes are called which i couldn't find in my list eg. D2, DA, E2 (i've compared some of the first opcode descriptions with the handler code and i found no difference)

Note: between the TRAP_ApplicationLayer blocks TRAP_Finished is invoked so the content code is idle. I don't know how long the java code runs

Edit: i'm looking at this (http://java.sun.com/docs/books/jvms/second_edition/html/Mnemonics.doc.html) list. Maybe it's incomplete.

loo3aem3ON

24th January 2009, 20:43

I was able to show that the java code which is interacting with the content code in the above described manner is located on the same disc (as expected). I've uploaded the previously mentioned java archive here (http://uploaded.to/?id=x64p86) for analysis. The first access (setting of register PSR104 to zero) takes placed through the call at "com/macrovision/bdplus/c/a+58"

Sadly most of the function names have been obfuscated. You don't need any special disassembler because it's just normal java code.

Rupan

24th January 2009, 21:48

I was able to show that the java code which is interacting with the content code in the above described manner is located on the same disc (as expected). I've uploaded the previously mentioned java archive here (http://uploaded.to/?id=x64p86) for analysis. The first access (setting of register PSR104 to zero) takes placed through the call at "com/macrovision/bdplus/c/a+58"

Sadly most of the function names have been obfuscated. You don't need any special disassembler because it's just normal java code.

I have experience reverse engineering java binary classes. I don't have time to work on it today, but I will try to commit source classes to svn tomorrow.

EDIT: it looks like class c may be the entry point for everything else, as you describe. It references many of the other classes (BDClassLoaderException, BDPlusSocketException, Handshake, LibHandshakeException, a, e, f, g, i, k). I don't have specific details at this time, but class c opens files named 77771.jar and mk/MK.enc, both of which may well be of interest to have uploaded somewhere. Further, the Handshake class uses 00000.otf from the archive root somehow -- but I haven't analyzed Handshake in-depth yet. One of the smaller classes appears to act as a container for 4 32-bit words, whose use I do not yet understand. Also, there are references to "BD-J" in strings in at least one class.

EDIT2: exception handlers make analysis harder, but many of the classes do not use them. I will commit source for those without exception handling tomorrow.

EDIT3: We need to find package org.bluray.bdplus.*. jar 7770 directly imports classes from this package and will not build without it. 77771.jar and 77772.jar are referenced in several source files. I'm not sure what to make of this, but a string in Handshake seems to suggest that 00000.otf is a "font" file. Whether or not this is intended as indirection is unclear.

EDIT4: it turns out that 00000.otf really is a font ... specifically, an Open Type Font file (which is apparently the successor to TrueType). The font is Bitstream Vera Sans Mono release 1.10.

loo3aem3ON

25th January 2009, 11:50

EDIT3: We need to find package org.bluray.bdplus.*. jar 7770 directly imports classes from this package and will not build without it. 77771.jar and 77772.jar are referenced in several source files. I'm not sure what to make of this, but a string in Handshake seems to suggest that 00000.otf is a "font" file. Whether or not this is intended as indirection is unclear.
I've uploaded the JAR directory from the disc and the BDJ.JAR from the player here (http://uploaded.to/?id=51tw3r). Let me know if you need anything else. Thanks for looking into this.

Rupan

25th January 2009, 14:01

You can pick up jad to accelerate reverse engineering. Jad is free (as in beer) software that decompiles java bytecode. Yes, you heard me -- decompiles. No more guessing at pesky java opcodes... This software will produce java source code from bytecode, which is in many cases compilable out of the box.

http://www.kpdus.com/jad.html

It isn't perfect, but it sure beats poring over bytecode disassembly.

loo3aem3ON

25th January 2009, 16:17

I've extracted the BDJ.JAR and the 77770.jar into the same directory. But trying to run the code fails:
$ java com/macrovision/bdplus/Handshake
Exception in thread "main" java.lang.NoSuchMethodError: main
And indeed none of the jar files contains a class with a "main" function. How is the code executed? I noticed that the *.bdjo files in the BDJO directory are referencing the class files in the jar. That's the 07777.bdjo:
0000:0000 42 44 4a 4f 30 32 30 30 00 00 00 30 00 00 00 3e BDJO0200...0...>
0000:0010 00 00 00 5c 00 00 00 64 00 00 01 3a 00 00 01 3e ...\...d...:...>
0000:0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0000:0030 00 00 00 0a 2a 2a 2a 2a 2a 1c 00 00 00 00 00 00 ....*****.......
0000:0040 00 1a 02 00 01 37 37 37 37 30 2a 2e 2a 00 00 00 .....77770*.*...
0000:0050 01 39 39 39 30 30 2a 2e 2a 00 00 00 00 00 00 04 .99900*.*.......
0000:0060 00 10 00 00 00 00 00 d2 02 00 01 10 7f ff 64 6c .......Ò.....ÿdl
0000:0070 77 77 00 00 00 00 00 64 00 00 00 00 10 00 00 01 ww.....d........
0000:0080 01 00 00 00 01 d0 00 0d 65 6e 67 09 42 44 53 74 .....Ð..eng.BDSt
0000:0090 75 62 58 6c 74 00 00 00 00 00 05 37 37 37 37 30 ubXlt......77770
0000:00a0 00 00 20 63 6f 6d 2e 6d 61 63 72 6f 76 69 73 69 .. com.macrovisi
0000:00b0 6f 6e 2e 62 64 70 6c 75 73 2e 48 61 6e 64 73 68 on.bdplus.Handsh
0000:00c0 61 6b 65 00 17 16 2d 42 44 2b 58 4c 45 54 3a 46 ake...-BD+XLET:F
0000:00d0 69 72 73 74 50 6c 61 79 58 6c 65 74 02 10 7f ff irstPlayXlet...ÿ
0000:00e0 64 6c 40 00 00 00 00 00 00 50 00 00 00 00 10 00 dl@......P......
0000:00f0 00 01 01 00 00 00 02 d0 00 11 65 6e 67 0d 46 69 .......Ð..eng.Fi
0000:0100 72 73 74 50 6c 61 79 58 6c 65 74 00 00 00 00 00 rstPlayXlet.....
0000:0110 05 39 39 39 30 30 00 00 1e 63 6f 6d 2e 62 79 64 .99900...com.byd
0000:0120 65 6c 75 78 65 2e 66 6f 78 2e 46 69 72 73 74 50 eluxe.fox.FirstP
0000:0130 6c 61 79 58 6c 65 74 00 00 00 00 00 00 00 00 08 layXlet.........
0000:0140 42 44 4d 56 2f 4a 41 52 BDMV/JAR

Edit: BD-J introduction: link (http://www.blueboard.com/bluray/introduction_bdj.htm)

Rupan

25th January 2009, 22:03

I've attached source code built using jad and edited by me for clarity and to compile. Some of the classes aren't fully fixed and the archive as a whole will not build, but it should give some idea of what is going on. It does appear that the jar includes more than bdplus code, so reversing all the classes may be a colossal waste of time. Better to understand the specific parts that do the bits that are interesting to us and reimplement them.

KenD00

25th January 2009, 22:25

That BD-J introduction is quite informative, thanks. As you probably have figured out yourself BD-J applications are applets and don't have a main method, they are controlled through the applet interface methods. So the entry point of the jar file is the Handshake class. That class name can be found in the bdjo file too, maybe this file contains the AMT the introduction mentions?

Running the applet won't get you far, you don't have the execution environment it requires. But this can be emulated, but i don't know how complicated that is. One big advantage is that everything is java, we have already our command interpreter (standard JRE) and we have our class library (provided by the player), we only need the player hardware. I have taken a very quick look at the BDJ.jar file, found the methods that access the BD+ PSR's. There are java methods that allow to read and write PSR's, these methods themselves call native methods which actually access the hardware. A good start for an emulator would be to extract all native methods and try to implement some of them. I will try to do the former when i have more time to get an overview.

:rolleyes:

loo3aem3ON

25th January 2009, 23:10

Thanks Rupan. I'll take a look at your work once the upload has been approved.

Running the applet won't get you far, you don't have the execution environment it requires. But this can be emulated, but i don't know how complicated that is.
I'm just playing around a bit to get a feeling of how it works. Maybe we can implement a simple environment to get this code running properly.

I've downloaded the Java Micro Edition "Personal Basis Profile 1.1 Reference Implementation (JSR 217)" from this (http://java.sun.com/javame/downloads/index.jsp) site now. If you want to try this be careful to keep the directory structure and run the cvm like "./bin/cvm". If you only copy the binary (intel-i386 code btw) and run it you will get a pretty useless error message.

wikipedia has a few interesting details too:
BD-J, or Blu-ray Disc Java, is a specification supporting Java ME (specifically the Personal Basis Profile of the Connected Device Configuration or CDC) Xlets for advanced content on Blu-ray Disc and the Packaged Media profile of Globally Executable MHP (GEM).

I have taken a very quick look at the BDJ.jar file, found the methods that access the BD+ PSR's. There are java methods that allow to read and write PSR's, these methods themselves call native methods which actually access the hardware.
Yes these functions were my "entry point". I just followed the returns back to the com/macrovision/bdplus/c class :rolleyes:

A good start for an emulator would be to extract all native methods and try to implement some of them. I will try to do the former when i have more time to get an overview.
I know the native code called by the setPSR() and getPSR() functions but i haven't studied the rest of the "interface" yet. It's probably a good idea to follow the "invokestatic" java instructions.

Accident

26th January 2009, 02:39

I took a glance at their implementation of RunNative, and most of which we already know.

uint32_t trap_RunNative(uint8_t *signature, uint32_t sigLen, uint8_t *code,
uint32_t codeLen)

Test of "signature" and "code" follow the regular "valid_ptr" tests, segLen should be 0x28 in length. codeLen fails if < 1. It then calls SHA on "code" (?) followed by ecdsa_verify, if that is ok it finally calls RunNative. What is peculiar is that there is a code-path that skips SHA and ecdsa_verify calls and directly calls RunNative.

RunNative itself is fairly boring, calls mkstemp("/tmp/run_native.XXXXXX"); write( , code, codeLen); chmod 0700; fork; execv / waitpid. The exit-code of the child process is eventually returned. RunNative itself blocks until script completes, although technically, scripts could easily put themselves in the background if so wanted. Scripts are run as player user, in this case root.

Kraise

26th January 2009, 06:06

Wow, I think that would work

loo3aem3ON

27th January 2009, 11:49

The Handshake between BD-J and BD+ involves AES but i haven't studied this in detail yet.

I found a stack trace:
at sun.security.util.DerValue.init(Unknown Source)
at sun.security.util.DerValue.<init>(Unknown Source)
at com.cl.bdj.jar.BDJSecurityUtil.getPermRootCertDigest(Unknown Source)
at com.cl.bdj.appmanager.BDJAppProxy.loadResources(Unknown Source)
at com.cl.bdj.appmanager.BDJAppProxy.startBDJApp(Unknown Source)
at com.cl.bdj.appmanager.BDJAppManager.runAutoStartApps(Unknown Source)
at com.cl.bdj.appmanager.BDJAppManager.startTitle(Unknown Source)
at com.cl.bdj.appmanager.BDJAppManager.initTitle(Unknown Source)
at com.cl.bdj.main.BDJCFacade.initTitle(Unknown Source)
According to this execution starts in the com.cl.bdj.main.BDJCFacade class which then calls the application manager which then starts the BD-J Application Proxy which finally starts the application (which is marked as "Autostart").

All the code is inside the BDJ.jar you can download here (http://uploaded.to/?id=51tw3r)

loo3aem3ON

27th January 2009, 19:55

It's my current understanding that the BD-J implementation consists of at least 3 major parts:

1. BD-J core implemented in java (vendor specific)
2. BD-J player interface (vendor specific) implemented in native code
3. common libraries implemented in java: javatv, GEM/MHP, ... (overview (http://java.sun.com/developer/technicalArticles/javame/bluray/#3))

If we use all the jar packages from one vendor we would only have to implement the native interface. The interface is probably vendor specific too.

The source code of the virtual machine interpreter (Personal Basis Profile of the Connected Device Configuration) is available from the phoneme (https://phoneme.dev.java.net/downloads_page.html#advanced) download page.

The undefined opcodes (eg. 0xDA) i saw during execution are generated by the interpreter during runtime. In other words the java bytecode is transformed during execution which is really confusing.

OwenRW

28th January 2009, 16:28

If anyone finds a description for some of the remaining unknown registers please post it here.

According to these...
http://www.wipo.int/pctdb/en/wo.jsp?wo=2006078111&IA=WO2006078111&DISPLAY=DESC
http://www.wipo.int/pctdb/en/wo.jsp?IA=KR2006003273&wo=2007024075&DISPLAY=DESC
http://www.freepatentsonline.com/y2007/0092223.html

Here are a couple more PSRs:

PSR30 Player Capability for Text Subtitle
PSR31 Player Version information
PSR32 Network Configuration
PSR33 Local Storage Size
PSR34 Streaming Media Capability

Hope that's useful for completing the list, if nothing else.

loo3aem3ON

28th January 2009, 16:50

Hope that's useful for completing the list, if nothing else.
Thank you. I've updated the description. :thanks:

loo3aem3ON

1st February 2009, 14:12

That's a build snapshot of the upcoming BD-J implementation: http://uploaded.to/?id=14h79o

The current debug log: [BDRootScene.getRootScene()] BDRootScene.notifyDirty
[HGraphicsHelper.init()] m_image.getWidth 1920
[HGraphicsHelper.init()] m_image.getHeight 1080
[HGraphicsHelper.init()] m_image.type 1
JMFProperties: readProperties
[AppsDatabaseHelper.getInitHAVIResolution()] [ERROR!]unknown initial HAVI config id, set as 1920x1080
[BDRootScene.getRootScene()] BDJPluginImp.toolkitSync
[BDRootScene.getRootScene()] BDRootScene.notifyDirty
[BDRootScene.getRootScene()] BDJPluginImp.toolkitSync
[BDJCFacade.executeCommand()] Start init Title
java.lang.NoSuchMethodException: java.util.jar.JarFile.extractJar(java.security.cert.X509Certificate, java.lang.String, [Z)
at java.lang.Class.getDeclaredMethod(Class.java:1937)
at com.cl.bdj.jar.JarAuthenication.initialReflections(Unknown Source)
at com.cl.bdj.jar.JarAuthenication.<clinit>(Unknown Source)
at com.cl.bdj.appmanager.BDJAppManager.checkRootCerts(Unknown Source)
at com.cl.bdj.appmanager.BDJAppManager.initTitle(Unknown Source)
at com.cl.bdj.main.BDJCFacade.initTitle(Unknown Source)
at com.cl.bdj.main.BDJCFacade.executeCommand(Unknown Source)
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJCFacade.executeCommand()] End init Title

Edit: error fixes. updated console output. It creates a graphic window on my screen now.

Edit: further bugfixes. Initialization succeeds. Problems after "init title" command (loading the application jar files from the disc).

Edit: updated debug log; new build snapshot

loo3aem3ON

1st February 2009, 14:13

The reference debug log from the player is:created one file output stream : javaLog.txt

ä¸*æ–‡Log started: Fri Jan 30 11:22:50 GMT-08:00 2009
[BDRootScene.getRootScene()] BDRootScene.notifyDirty
[HGraphicsHelper.init()] m_image.getWidth 1920
[HGraphicsHelper.init()] m_image.getHeight 1080
[HGraphicsHelper.init()] m_image.type 2
JMFProperties: readProperties
[AppsDatabaseHelper.getInitHAVIResolution()] [ERROR!]unknown initial HAVI config id, set as 1920x1080
[BDRootScene.getRootScene()] BDJPluginImp.toolkitSync
[BDRootScene.getRootScene()] BDRootScene.notifyDirty
[BDRootScene.getRootScene()] BDJPluginImp.toolkitSync
[BDRootScene.checkGraphicsPlaneSize()] [ERROR!][resolution] screen size unknown
[BDJCFacade.executeCommand()] Start init Title
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.macrovision.bdplus.Handshake
[BDJCFacade.executeCommand()] End init Title
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doInitXlet()] init BD-J app : com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doInitXlet()] init BD-J app ended: com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doStartXlet()] start BD-J app : com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doStartXlet()] start BD-J app ended: com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doInitXlet()] init BD-J app : com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doInitXlet()] init BD-J app ended: com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doStartXlet()] start BD-J app : com.bydeluxe.fox.FirstPlayXlet
com.cl.bdj.appmanager.BDJThreadGroup[name=BDJThreadGroup-com.macrovision.bdplus.Handshake,maxpri=5]
[RegisterAccess.setGPR()] setGPR(1009)=0
[RegisterAccess.setGPR()] setGPR(1008)=0
[RegisterAccess.setGPR()] setGPR(1007)=0
[RegisterAccess.setGPR()] setGPR(1006)=0
[RegisterAccess.setGPR()] setGPR(1005)=0
[RegisterAccess.setGPR()] setGPR(1004)=0
[RegisterAccess.setGPR()] setGPR(1003)=0
[RegisterAccess.setGPR()] setGPR(1002)=0
[RegisterAccess.setGPR()] setGPR(1001)=0
[RegisterAccess.setGPR()] setGPR(1000)=0
[RegisterAccess.setGPR()] setGPR(1030)=0
[RegisterAccess.setGPR()] setGPR(1029)=0
[RegisterAccess.setGPR()] setGPR(1028)=0
[RegisterAccess.setGPR()] setGPR(1027)=0
[RegisterAccess.setGPR()] setGPR(1026)=0
[RegisterAccess.setGPR()] setGPR(1025)=0
[RegisterAccess.setGPR()] setGPR(1024)=0
[RegisterAccess.setGPR()] setGPR(1023)=0
[RegisterAccess.setGPR()] setGPR(1022)=0
[RegisterAccess.setGPR()] setGPR(1021)=0
[RegisterAccess.setGPR()] setGPR(1020)=0
[RegisterAccess.setGPR()] setGPR(1019)=0
[RegisterAccess.setGPR()] setGPR(1018)=0
[RegisterAccess.setGPR()] setGPR(1017)=0
[RegisterAccess.setGPR()] setGPR(1016)=0
[RegisterAccess.setGPR()] setGPR(1015)=0
[RegisterAccess.setGPR()] setGPR(1014)=0
[RegisterAccess.setGPR()] setGPR(1013)=0
[RegisterAccess.setGPR()] setGPR(1012)=0
[RegisterAccess.setGPR()] setGPR(1011)=0
[RegisterAccess.setGPR()] setGPR(1010)=0
[RegisterAccess.getPSR()] getPSR(20)=2
[RegisterAccess.getPSR()] getPSR(13)=255
[RegisterAccess.getPSR()] getPSR(31)=197120
[BDJSelectTitleEvent.callEvent()] [Begin] select title: 2
[BDJEventTicket.waitForEventStart()] [Start] process event com.cl.bdj.event.BDJEventTicket@478783
[BDJAppProxy.destroyBDJApp()] destroyBDJApp -- the app is not loaded
[BDJPrimitiveCommand.doStartXlet()] start BD-J app ended: com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app : com.bydeluxe.fox.FirstPlayXlet
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app ended: com.bydeluxe.fox.FirstPlayXlet
[BDJCommandRunner.forceExit()] the runner: BDJCommandRunner:Name<PaintRunner>,Owner<com.cl.bdj.appmanager.MainXletContext@52bc4e( BDJAppProxy (2147443820, 16384, com.bydeluxe.fox.FirstPlayXlet) )> was not stopped
com.cl.bdj.appmanager.BDJThreadGroup[name=BDJThreadGroup-com.bydeluxe.fox.FirstPlayXlet,maxpri=5]
Thread[AWTEventQueueThread-com.bydeluxe.fox.FirstPlayXlet,5,BDJThreadGroup-com.bydeluxe.fox.FirstPlayXlet]
Thread[BDJPaintRunnerThread-com.bydeluxe.fox.FirstPlayXlet,5,BDJThreadGroup-com.bydeluxe.fox.FirstPlayXlet]
[BDJCFacade.executeCommand()] Start init Title
[BDJAppProxy.destroyBDJApp()] destroyBDJApp -- the app is not loaded
[AppsDatabase.getAppProxy()] Input AppID:(1)getOID()=2147443820,(2)getAID()=30583
[BDJAppProxy.sendStateChangeEvent()] App-orgID: 2147443820, App-appID: 30583 => getAppProxy(appID) returns null
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJCFacade.executeCommand()] End init Title
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.bydeluxe.fox.BDLiveReadyXlet
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.bydeluxe.fox.BDLiveReadyXlet
[BDJPrimitiveCommand.doInitXlet()] init BD-J app : com.bydeluxe.fox.BDLiveReadyXlet
[BDJSelectTitleEvent.callEvent()] [End] select title: 2
[BDJPrimitiveCommand.doInitXlet()] init BD-J app ended: com.bydeluxe.fox.BDLiveReadyXlet
[BDJPrimitiveCommand.doStartXlet()] start BD-J app : com.bydeluxe.fox.BDLiveReadyXlet
[BDJPrimitiveCommand.doStartXlet()] start BD-J app ended: com.bydeluxe.fox.BDLiveReadyXlet
[DiscFontManager.getFontIndexData()] fontindexDataBuffer is null
[HScene.reloadDefaultFont()] SetDefaultFont: style(0) Size(26)
Creating buffer: 1920 x 1080
, 8294400 bytes
1 images. 8294400bytes
https://0000-0000-1D6F-0000-5-0000-0003-G.foxbd-live.com/BDLiveServices/Service.asmx/BootLoader
VolumeID: 6C2C3851-D4BF-39AC-E53B-8EA78E866E45
[RegisterAccess.getPSR()] getPSR(18)=6647399
class java.lang.SecurityException Doesn't contain valid SocketPermission
java.lang.SecurityException: Doesn't contain valid SocketPermission
at com.cl.bdj.jar.BDJSecurityManager.checkPermissionImp(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager.access$000(Compiled Method)(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkConnect(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName(Unknown Source)
at java.net.InetAddress.getByName(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getHeaderFieldKey(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderFieldKey(Unknown Source)
at ae.a(Unknown Source)
at w.k(Unknown Source)
at w.run(Unknown Source)
at java.lang.Thread.startup(Unknown Source)
java.lang.SecurityException: Doesn't contain valid SocketPermission
at com.cl.bdj.jar.BDJSecurityManager.checkPermissionImp(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager.access$000(Compiled Method)(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Unknown Source)
at com.cl.bdj.jar.BDJSecurityManager.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkConnect(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName(Unknown Source)
at java.net.InetAddress.getByName(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.a(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getHeaderFieldKey(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderFieldKey(Unknown Source)
at ae.a(Unknown Source)
at w.k(Unknown Source)
at w.run(Unknown Source)
at java.lang.Thread.startup(Unknown Source)
Continue Normal Disc Execution
Removing 8294400 bytes.
0 images. 0bytes
4
[BDJEventTicket.waitForEventStart()] [Start] process event com.cl.bdj.event.BDJEventTicket@22a11b
[BDJSelectTitleEvent.callEvent()] [Begin] select title: 4
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app : com.bydeluxe.fox.BDLiveReadyXlet
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app ended: com.bydeluxe.fox.BDLiveReadyXlet
com.cl.bdj.appmanager.BDJThreadGroup[name=BDJThreadGroup-com.bydeluxe.fox.BDLiveReadyXlet,maxpri=5]
[BDJCFacade.executeCommand()] Start init Title
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJPlaneControlHelper.setScreenKeepResolution()] KeepResolution(false)
[BDJCFacade.executeCommand()] End init Title
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.bydeluxe.fox.OpeningSequenceXlet
[BDJSelectTitleEvent.callEvent()] [End] select title: 4
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.bydeluxe.fox.OpeningSequenceXlet
[BDJPrimitiveCommand.doInitXlet()] init BD-J app : com.bydeluxe.fox.OpeningSequenceXlet
[RegisterAccess.getPSR()] getPSR(18)=6647399
[RegisterAccess.getPSR()] getPSR(20)=2
[BDJPrimitiveCommand.doInitXlet()] init BD-J app ended: com.bydeluxe.fox.OpeningSequenceXlet
[BDJPrimitiveCommand.doStartXlet()] start BD-J app : com.bydeluxe.fox.OpeningSequenceXlet
Reading /VFS//BDMV/JAR/99801/opening.properties
[RegisterAccess.getPSR()] getPSR(16)=6647399
getPlaylistId interpol.eng.eng.B = null
getPlaylistId interpol.eng = -1
[RegisterAccess.getPSR()] getPSR(17)=6647399
getPlaylistId disclaimer.eng.eng.B = null
getPlaylistId disclaimer.eng = -1
getPlaylistId mpaa.eng.B = null
getPlaylistId mpaa = -1
getPlaylistId playlist1.eng.B = null
getPlaylistId playlist1 = 5
[RegisterAccess.getPSR()] getPSR(1)=1
[RegisterAccess.getPSR()] getPSR(2)=268374015
[BDJPrimitiveCommand.doStartXlet()] start BD-J app ended: com.bydeluxe.fox.OpeningSequenceXlet
[BDJCFacade.executeCommand()] ====== start do suspend ======
[BDJCFacade.executeCommand()] ====== end do suspend ======
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app : com.bydeluxe.fox.OpeningSequenceXlet
[BDJPlaneControlHelper.getAWTVideoSize()] [ERROR!]getPlaneControl return null
java.lang.Exception
at com.cl.bdj.helper.BDJPlaneControlHelper.getAWTVideoSize(Unknown Source)
at com.cl.bdj.media.control.AWTVideoSizeControlImpl.getSize(Unknown Source)
at com.cl.bdj.media.control.BDJPlaylistAWTVideoSizeControl.getSize(Unknown Source)
at com.cl.bdj.media.BDJPlayListPlayer.hookStop(Unknown Source)
at com.cl.bdj.media.BDJJMFPlayerImpl.stop(Unknown Source)
at c.j(Unknown Source)
at c.a(Unknown Source)
at com.bydeluxe.fox.OpeningSequenceXlet.destroyXlet(Unknown Source)
at com.cl.bdj.appmanager.BDJPrimitiveCommand.doDestroyXlet(Unknown Source)
at com.cl.bdj.appmanager.DestroyXletCommand.execute(Unknown Source)
at com.cl.bdj.event.BDJCommandRunner.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
at java.lang.Thread.startup(Unknown Source)
[BDJPrimitiveCommand.doDestroyXlet()] destroy BD-J app ended: com.bydeluxe.fox.OpeningSequenceXlet
[BDJCommandRunner.forceExit()] the runner: BDJCommandRunner:Name<AppRunner>,Owner<XletRunnerContext<null>> was not stopped
com.cl.bdj.appmanager.BDJThreadGroup[name=BDJThreadGroup-com.bydeluxe.fox.OpeningSequenceXlet,maxpri=5]
[BDJCFacade.exitAll()] end exitAll(), clear Image Cache
[BDJCFacade.exitAll()] Clear Image Cache
[BDJCFacade.clearImageCache()] Finish clear image cache
[BDJCFacade.flushAllImageData()] Finish flush image psd
CBDJMediaProcessor has been stopped or finalized!
CBDJMediaProcessor has been stopped or finalized!
CBDJMediaProcessor has been stopped or finalized!

loo3aem3ON

3rd February 2009, 14:26

I've uploaded a new binary snapshot (see posting #657 (https://forum.doom9.org/showpost.php?p=1244334&postcount=657)). The developers should try to get it running so we can quickly proceed when it executes the 77770.jar (BD+ handshake). I expect this to happen in the next view days.

To execute: (example)
export LD_LIBRARY_PATH=${HOME}/bdj/jdk1.6.0_11/jre/lib/i386/:${HOME}/bdj/jdk1.6.0_11/jre/lib/i386/server
./bdj /absolute_path/jni/libbdjinterface.so
Look for a logfile: ${HOME}/javaLog.txt

to compile the bdj.c: (example; optional)
gcc -m32 -L/${HOME}/bdj/jdk1.6.0_11/jre/lib/i386/server -ljvm -I/${HOME}/bdj/jdk1.6.0_11/include/ -o bdj bdj.c

to compile the interface (jni): (optional)
cd jni
make

KenD00

6th February 2009, 20:14

I am very low on free time currently so i couldn't test this one earlier. After adjusting the paths to fit my installation and installing libqt3-mt i still can't get it running. The problem is the following

Exception in thread "main" [W] java.lang.UnsatisfiedLinkError: no qtawt in java.library.path

I can't convince java to find that damn shared library. I tried moving it around, added a java.library.path definition to the vmargs inside bdj.c but it still won't find that damn library :confused:. In the beginning i had that problem with DumpHD too but it was fixed adjusting java.library.path but this doesn't help right now. I will experiment around a little more but maybe someone else had the same problem and knows a solution.

:rolleyes:

loo3aem3ON

6th February 2009, 21:22

Exception in thread "main" [W] java.lang.UnsatisfiedLinkError: no qtawt in java.library.path

I can't convince java to find that damn shared library. I tried moving it around, added a java.library.path definition to the vmargs inside bdj.c but it still won't find that damn library :confused:.
Maybe your libqt3-mt.so.3 is not 32bit or some other dependencies of libqtawt.so are not met. I had to install the 32bit version of libqt3-mt.so.3 manually because i work on an amd64 system. From my notes:
http://www.boundlesssupremacy.com/Cappy/getlibs/getlibs-all.deb
getlibs -w http://ftp.de.debian.org/debian/pool/main/q/qt-x11-free/libqt3-mt_3.3.8b-5+b1_i386.deb
I can't compile the libqtawt.so (closed source?) and only the 32-bit version is available.

Anyway i recently switched from Java SE to ME and compiled the compact virtual machine (cvm) with debug flag so it's pretty verbose if something goes wrong. You can grab my binary snapshot from here (http://uploaded.to/?id=a91uz7). Run it with (adjust the paths):
./bin/cvm ${HOME}/bdplus/trunk/bdj/jni/libbdjinterface.so
The snapshot contains a standalone Java (microedition) installation so it shouldn't use any foreign jar files or stuff from other java installations. My player uses the cvm too and it's part of the BD-J standard.

If you want to compile it you will need the phoneme advanced mr2 snapshot. I had to merge the bdj.c with the phoneme source before compiling the cvm. Let me know if you need to know more details.

KenD00

7th February 2009, 20:36

I'm running this on a 32 bit Ubuntu, ldd reports no missing libs. I've written a small java app which just loads the qtawt library and this works, don't know why it doesn't work from bdj.

Anyway, i've tried your cvm snapshot and had more success, qtawt is now loaded and i get a black window. During startup i got this error

"/home/<scrubbed>/bdj-cvm/lib/basis.jar" is not a valid classpath zip file or directory.

which i fixed by renaming basis_security.jar.

The cvm runs but fails to load any BD-J applet from the disc. The path to my drive matches the hardcoded path so this isn't the problem. I can't remember you ever mentioned which disc you are using, i assume we are talking about X-Files here because your posted 77770.jar matches almost the one from my disc (but there are differences, i think because i don't have your version of the movie and i remember Anydvd was updated to support a different version of X-Files), but i don't have 99900.jar (and no directory with that name) on my disc.

These are the error messages from loading the 99900.jar file, loading the 77770.jar produces the same ones:

[W] [BDJAppManager.startTitle()] startTitle(65535)
[W]

[W] [BDJAppManager.setTitleSelectState()] set current title context state: 3, previos state: 1
[W]

called Java.com.cl.bdj.helper.AppsDatabaseHelper.pGetAppControlCode
[W] java.util.zip.ZipException: No such file or directory
[W]

[W] at java.util.zip.ZipFile.open(Ljava/lang/String;IJ)J(Native Method)
[W]

[W] at java.util.zip.ZipFile.<init>(Ljava/io/File;I)V(ZipFile.java:146)
[W]

[W] at java.util.jar.JarFile.<init>(Ljava/io/File;ZI)V(JarFile.java:138)
[W]

[W] at java.util.jar.JarFile.<init>(Ljava/lang/String;Z)V(JarFile.java:90)
[W]

[W] at com.cl.bdj.jar.JarAuthenication.extractJar(Ljava/lang/String;Ljava/lang/String;Z)V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppProxy.securityCheckProcess(Ljava/lang/String;Ljava/lang/String;)V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppProxy.loadResources()V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppProxy.startBDJApp([Ljava/lang/String;)V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppManager.runAutoStartApps()V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppManager.startTitle(I)V(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJAppManager.initTitle(Z)V(Unknown Source)
[W]

[W] at com.cl.bdj.main.BDJCFacade.initTitle(I)V(Unknown Source)
[W]

[W] at com.cl.bdj.main.BDJCFacade.executeCommand(I[Ljava/lang/Object;)I(Unknown Source)
[W]

[W] [BDJAppProxy.loadResources()] Start printing class path for application:OID(2147443820), AID(16384)
[W]

[W] [BDJAppProxy.loadResources()] ClassPath<0>:file:///DOCR/signed/99900/
[W]

called Java.com.cl.bdj.helper.AppsDatabaseHelper.pGetAppBaseDirectory -> OrgID = 7FFF646C, AppID = 4000
called Java.com.cl.bdj.helper.AppsDatabaseHelper.pGetAppClasspathExt -> OrgID = 7FFF646C, AppID = 4000
[W] [BDJAppUtil.getAppClasspath()] /DOCR/signed/
[W]

[W] [BDJAppUtil.getAppClasspath()] 99900
[W]

[W] [BDJAppUtil.getAppClasspath()]
[W]

[W] [BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.bydeluxe.fox.FirstPlayXlet
[W]

called com.cl.bdj.helper.BDJTitleInfoHelper.pGetCurrentDiscIdFromC
called Java.com.cl.bdj.helper.CUtil.pSetJITOffSwitch -> NOT IMPLEMENTED!
[W] [BDJClassLoader.findClass()] Can't find class:com.bydeluxe.fox.FirstPlayXlet, start printing class path ...
[W]

[W] [BDJClassLoader.findClass()] ClassPath<0>:file:/DOCR/signed/99900/
[W]

[W] [BDJAppUtil.createXletInstance()] [ERROR!]in createXletInstance(), caught exception ClassNotFoundException
[W]

[W] java.lang.ClassNotFoundException: com.bydeluxe.fox.FirstPlayXlet
[W]

[W] at java.net.URLClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class;(URLClassLoader.java:234)
[W]

[W] at com.cl.bdj.appmanager.BDJClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class;(Unknown Source)
[W]

[W] at java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;(ClassLoader.java:349)
[W]

[W] at java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;(ClassLoader.java:291)
[W]

[W] at com.cl.bdj.appmanager.BDJAppUtil.createXletInstance(Ljava/lang/ClassLoader;Ljava/lang/String;)Ljava/lang/Object;(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.BDJPrimitiveCommand.doLoadXlet(Lcom/cl/bdj/appmanager/XletRunnerContext;)Z(Unknown Source)
[W]

[W] at com.cl.bdj.appmanager.StartXletCommand.execute()V(Unknown Source)
[W]

[W] at com.cl.bdj.event.BDJCommandRunner.run()V(Unknown Source)
[W]

[W] at java.lang.Thread.run()V(Thread.java:712)
[W]

[W] at java.lang.Thread.startup(Z)V(Thread.java:782)
[W]

[W] [BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.bydeluxe.fox.FirstPlayXlet
[W]

[W] [BDJAppProxy.stateTransition()] state change failed : BDJAppProxy (2147443820, 16384, com.bydeluxe.fox.FirstPlayXlet), 2 -> 0
[W]

loo3aem3ON

7th February 2009, 23:42

The cvm runs but fails to load any BD-J applet from the disc.
It doesn't load the jar files from the disc. Instead you have to copy them to "/tmp/BDJRoot/" (both the 77770.jar and 99900.jar). That directory is the local application cache which can change over time when using BD-Live. It will extract a part of the jar archives inside the "DOCR/signed/" subdirectory. I don't fully understand this yet.

I can't remember you ever mentioned which disc you are using, i assume we are talking about X-Files here because your posted 77770.jar matches almost the one from my disc (but there are differences, i think because i don't have your version of the movie and i remember Anydvd was updated to support a different version of X-Files), but i don't have 99900.jar (and no directory with that name) on my disc.
It's "The Day The Earth Stood Still" and you can download the content of the JAR directory from posting #645 (https://forum.doom9.org/showpost.php?p=1241632&postcount=645).

These are the error messages from loading the 99900.jar file, loading the 77770.jar produces the same ones:
I'm glad to see you made it this far. You should get a bit further with both jar files in the /tmp/BDJRoot/ directory but there will be null pointer exception (certificate == null?) soon afterwards.

A good way to proceed is to edit the com.cl.bdj.helper.* classes (inside the bdj.jar) to make them print out a console message of the actual value/string they got from the native methods they called. For example:
invokestatic com/cl/bdj/util/BDJTrace/info(Ljava/lang/String;)V
will print the string you pass as parameter plus the class and method name from which it is called.
The additional debug output i get when using the modified bdj.jar helps me to correct the bugs in the native interface. My source code C files are named after the helper classes as you might have noticed.

loo3aem3ON

14th February 2009, 17:44

The classes in the "btclasses.zip" archive have been merged into the cvm binary (ROMization) to boost performance. That causes some problems because some of these classes have been customized by the vendor of my player. For example a "extractJar" method has been added to the java.util.jar.JarFile which is therefor missing in the cvm i compiled. I have several options in this case and will try them all.
I also managed to patch all methods which call native code and return boolean, int or string. I also succeeded in attaching a debugger (jdb) to the cvm to monitor the BD-J implementation. It's quite complex.

Progress is slow because i'm working alone on this. So be patient.

loo3aem3ON

15th February 2009, 19:58

I finally got the BD+ handshake application loaded and running:
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app : com.macrovision.bdplus.Handshake
[...]
[BDJPrimitiveCommand.doLoadXlet()] load BD-J app ended : com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doInitXlet()] init BD-J app : com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doInitXlet()] init BD-J app ended: com.macrovision.bdplus.Handshake
[BDJPrimitiveCommand.doStartXlet()] start BD-J app : com.macrovision.bdplus.Handshake
[...]
[BDJPrimitiveCommand.doStartXlet()] start BD-J app ended: com.macrovision.bdplus.Handshake
[BDJAppProxy.stateTransition()] state change succeeded : BDJAppProxy (2147443820, 30583, com.macrovision.bdplus.Handshake), 2 -> 0
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0) -> NOT IMPLEMENTED!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, F1000000) -> NOT IMPLEMENTED!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change -> NOT IMPLEMENTED!
The application (com.macrovision.bdplus.Handshake) sets PSR103 to 0xF1000000 and PSR104 to 0 as expected (see first lines of posting #642 (https://forum.doom9.org/showpost.php?p=1241353&postcount=642)). The following call "pTriggerPSR103change" probably places a trigger on PSR103 so the thread resumes once the content code is done with its changes. I don't know the exact mechanism yet.

You can download the binary snapshot here (http://uploaded.to/?id=q83orr).

Steps to reproduce:
1. mount the disc at "/media/cdrom/" (certificate will be verified)
1. place the 77770.jar and the 99900.jar in the directory "/tmp/BDJRoot/" (see posting #645 (https://forum.doom9.org/showpost.php?p=1241632&postcount=645))
2. create a directory "/DOCR/" (root directory -> i couldn't move it into the "/tmp/" folder)
3. make sure both directories have user write permission (stuff will be written into both directories by the java code)
4. run: "./bin/cvm ${HOME}/bdplus/trunk/bdj/jni/libbdjinterface.so" <-- adapt to your directory structure

Edit: the "bdj.c" in the archive is slightly different from the version i used to create the cvm (debugging with jdb is currently disabled).

loo3aem3ON

16th February 2009, 02:06

After the content code wrote PSR102 the player executes command 8 (native code calls java method "executeCommand") which creates a BDPlusStatusEvent object. The constructor gets the contents of PSR102 as parameter. The status event object is then passed to the application. The "pTriggerPSR103change" call might cause BD+ event#0210.

I'll run some tests to see if the handshake works properly. If successful i can start merging this into libbluray. :)

loo3aem3ON

18th February 2009, 01:51

It seems to be working. Below are the relevant sections of the console output. You should compare this with posting #642 (https://forum.doom9.org/showpost.php?p=1241353&postcount=642) (look for the correct return "BC79595F 97F05643 07A0ABF2 CC10F7C5")
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0x0)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0xF1000000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> FFFFFFFE!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x4000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 3EA9DAFF!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x8000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> B23259D5!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0xC000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 53B07B58!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x10000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 60FBC4BB!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x14000)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 60FBC4BC!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0x60FBC4BD)
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0xBC79595F)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x4014004)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0x97F05643)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x4014008)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0x7A0ABF2)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x401400C)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0xCC10F7C5)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x4014010)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 10000000!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x18010)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 00000000!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(103, 0x1A010)
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pTriggerPSR103change
[...]
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pGetPSR(104) -> 00000001!
called Java.com.cl.bdj.helper.BDJVFSHelper.RegisterAccessHelper.pSetPSR(104, 0x2)
So we should probably start creating a libbdj which can be used in conjunction with libbluray to repair the latest batch of BD+ corrupted discs.

Btw the handshake application needs additional jar files but these are fetched automatically from the disc after i changed the VFS root to /media/cdrom. :)

br0kenpipe

18th February 2009, 13:35

You're doing an excellent job loo3aem3ON! And I would gladly help. But I'm afraid I don't know much about Java, and pretty much anything else discussed in this thread. ;)

Just wanted to give you an encouraging word, and keep up the excellent work!

loo3aem3ON

18th February 2009, 14:30

Thanks. You can help by grabbing the latest binary snapshot and playing around with it. Then report the problems you encounter.

About 80% of BD-J is implemented in java and this part can be copied from any Blu-Ray player without modifications. The rest are native functions which have to be implemented in C/C++ or whatever. Of particular interest are the functions which are used to indirectly talk with the BD+ content code via the player status registers (PSR) 102-104.

I don't know how difficult it is to get the menus working. A black window pops up in the top left corner of the screen but it isn't filled with contents because that's being done by the second application which i haven't yet tried to start. The primary goal was to get the first application (BD+ handshake) running.

slicemaster102

18th February 2009, 21:20

Thanks. You can help by grabbing the latest binary snapshot and playing around with it. Then report the problems you encounter.

About 80% of BD-J is implemented in java and this part can be copied from any Blu-Ray player without modifications. The rest are native functions which have to be implemented in C/C++ or whatever. Of particular interest are the functions which are used to indirectly talk with the BD+ content code via the player status registers (PSR) 102-104.

I don't know how difficult it is to get the menus working. A black window pops up in the top left corner of the screen but it isn't filled with contents because that's being done by the second application which i haven't yet tried to start. The primary goal was to get the first application (BD+ handshake) running.

This is great news, the progress you and the others working on this project have made in the last few weeks is more then the entire Linux community efforts combined for the last…o 6 moths or so. The guys here on the doom9 forums have almost successfully reverse-engineered the VM completely and now you are braking down the walls for BD-J in order to completely bypass BD+. An amazing feat if you ask me considering the relatively short amount of time you guys have been working on this. As to your comments regarding getting a full BD-J library in development, I feel this is the next logical step as it will be needed to get BD+ is fully handled. This is not to mention the fact that once we have an operational BD-J library, we would more or less have fully functional menu support under Linux with relatively little effort compared to what has already been done.

Keep up the good work guys!

Slice

P.S. something that might be nice to get more people testing this stuff is to make it easily and openly available, maybe precompiled and available for download from a repository for Linux. Also, basic step by step instructions on how to use these tools is not readily available so that might be something to consider.

P.P.S. I have not followed the progress of AACS very closely, has it been completely broken yet or are we still stuck only being able to decrypt titles that feature MKBv7 or earlier?

kkloster21

18th February 2009, 22:18

@ Slice:

as of now we (Linux users) are only able to decrypt titles that are MKBv8 and earlier. We don't have processing keys for MKBv's beyond that. Although Mike Chen of the MakeMKV project claims that they have keys that may be able to go as far as MKBv11:

http://forum.doom9.org/showthread.php?p=1242679#post1242679

but he hasn't yet responded to a request to share these keys with KenD00 (for DumpHD).

[edit: make that MKBv12: http://forum.doom9.org/showthread.php?p=1248998#post1248998]

KenD00

18th February 2009, 23:50

P.P.S. I have not followed the progress of AACS very closely, has it been completely broken yet

AACS most probably will never be broken and there will be always the hunt for new keys.

:rolleyes:

880

19th February 2009, 11:14

Can I donate money or blu-ray discs to you? I am not very technical but I want to help out.

Raybuntu

20th February 2009, 11:49

Hi,
where I can find the source code of this project. Maybe you could host this project on https://launchpad.net/ ? There you can make an bazaar repo so others could hack on the code too!? I could help you to register and host this project on Launchpad. I can also provide you help for Debian/Ubuntu packaging. I would be glad to helping this project :)

loo3aem3ON

20th February 2009, 13:02

in order to completely bypass BD+.
I don't think "bypassing" BD+ is possible. We are doing exactly what all the licensed Blu-Ray players are doing.

This is not to mention the fact that once we have an operational BD-J library, we would more or less have fully functional menu support under Linux with relatively little effort compared to what has already been done.
Hopefully :)

P.S. something that might be nice to get more people testing this stuff is to make it easily and openly available, maybe precompiled and available for download from a repository for Linux.
You can find full build/binary snapshots in my previous postings.

Also, basic step by step instructions on how to use these tools is not readily available so that might be something to consider.
Will do.

AACS most probably will never be broken and there will be always the hunt for new keys.
I would be a careful with such statements. Even if no weakness can be found (which i think is unlikely) the underlying AES cipher is losing 1 bit of its security every 18 months (Moore's law).
Well maybe by the time AES becomes insecure we will all have DRM modules implanted in our brains. :rolleyes:

Can I donate money or blu-ray discs to you? I am not very technical but I want to help out.
No thank you.

where I can find the source code of this project.
It's currently hosted at assembla: https://trac.assembla.com/bdplus/

Maybe you could host this project on https://launchpad.net/ ?
I'm currently quite happy with assembla but it might be a good idea to move the BD-J part (trunk/bdj) to a different repository.

I would be glad to helping this project :)
AACS needs to added to libbluray (currently BD+ only). The maintainer (Accident) is busy at the moment. You could help by finding volunteers to help with the project.
I need to explain and simplify the environment for BD-J so more people are willing to experiment. The directory structure needs to be explained (eg. BUDA = "Binding Unit Data Area" etc). Furthermore some of the native functions return data specific to the movie "The Day The Earth Stood Still".

loo3aem3ON

21st February 2009, 21:43

The BD-J platform development project is now hosted at https://launchpad.net/bdj.

The method "readBDJO" (trunk/src/bdjo/BDJOReader.cpp) and its dependencies need to be ported from java to C++. I've already renamed all the file extensions. The conversion from and to XML is irrelevant. Have a look at the method names in "AppsDatabaseHelper.c" and you will see why.

The native code first needs to parse the index.bdmv to obtain the FirstPlaybackmobjidref (name of a BD-J object in the BDJO/ directory) and then parse the identified *.bdjo file.
At the moment the data from the 07777.bdjo is hardcoded in the methods of "AppsDatabaseHelper.c" so other applications from other Blu-Rays won't work. Furthermore the AppCacheInfo section of this BD-J object contains a list files which have to be loaded into the local cache (which i called "/tmp/BDJRoot/") for fast access. Currently these files have to be copied manually. So you see how important it is to read these BD-J objects properly.
Reading the index.bdmv is not a problem.

Accident

22nd February 2009, 06:55

AACS needs to added to libbluray (currently BD+ only). The maintainer (Accident) is busy at the moment. You could help by finding volunteers to help with the project.

I want to apologise not being of much help lately. Large equipment failure that I got the honour to fix. But I can do code just fine when we have new things to solve, but no time to help with the in-depth research. On related news, I actually ended up with a BD player and a couple of disks, so it will be fun to be able to some experience actually playing a disk.

Related to libaacs, and libbluray, I was thinking of keeping the libraries separated, but write some example sources that uses both libraries, and massage them to play together nicely. I may end up checking in libaacs since its maintainer appears to be MIA. I have version 0.05.

loo3aem3ON

22nd February 2009, 11:46

That's the latest binary/build snapshot containing the libraries missing from the launchpad repository: Download (http://uploaded.to/?id=qkqdrm).
Start it simply with "sh run.sh". The disc has to be mounted at "/media/cdrom". It probably won't run far so don't be disappointed (see my previous posting for an explanation). I'm working on it.

@Accident:
Focus on libaacs then and maybe rename libbluray to libbdplus.

lbalan79

22nd February 2009, 17:37

Is there any way to help test this code without the actual use of an blu-ray phisical reader? I do not have an blu-ray unit but I would gladly help in case there is any test work that needs to be done.

loo3aem3ON

22nd February 2009, 19:10

Is there any way to help test this code without the actual use of an blu-ray phisical reader?
It doesn't require the physical disc. You can copy these files/directories to /media/cdrom/ (path is currently hardcoded as well) :
CERTIFICATE/app.discroot.crt
CERTIFICATE/bu.discroot.crt
BDMV/index.bdmv
BDMV/BDJO/*
BDMV/JAR/*
None of these files are encrypted. At least one *.jar file has to be present in BDMV/JAR/ otherwise there is no application to execute.

See if you can find these files somewhere. Otherwise i could send you those from my disc.

880

23rd February 2009, 08:25

In the readme for BDVMDebug 0.1.5 it says to post the conversion table on the forum. Should I post conv_tab.bin here? Should I post console output?

loo3aem3ON

23rd February 2009, 11:15

In the readme for BDVMDebug 0.1.5 it says to post the conversion table on the forum. Should I post conv_tab.bin here? Should I post console output?
The readme is outdated but you still can upload the conversion table somewhere if you want to. Have you tried to view the table with ConvTableView? Are there any console messages?

loo3aem3ON

25th February 2009, 00:25

I've started a new thread for BD-J related discussions: link (https://forum.doom9.org/showthread.php?t=145249)
Rupan has imported Brian Gladman's AES implementation into libbluray which needs testing. I'll remove the BD-J stuff from the assembla repository soon. It's now available on launchpad (link (https://launchpad.net/bdj)).

loo3aem3ON

4th March 2009, 16:55

Is someone willing to connect libbdjinterface and libbluray or do i have to do this myself too? Both libraries access the same PSR and GPR.

Remember:
PSR102 = BD+ to BD-J pipe
PSR103 = BD-J to BD+ pipe
PSR104 = BD-J/BD+ shared register
The call of pTriggerPSR103change means that the BD-J program has sent a message (PSR103) to the content code. The content code needs to handle this as EVENT#0210 next.
If in turn the content code writes PSR102 then EVENT#008 is passed to the BD-J code (see last view lines of bdj.c for an example).

Accident

5th March 2009, 05:44

I can probably squeeze that in over the weekend, at least I can make an effort to.

loo3aem3ON

6th March 2009, 14:42

@Rupan: Could you please explain your changes to TRAP_Aes and TRAP_SHA? What are the advantages of Brian Gladman's AES and SHA-1 implementations?

Rupan

6th March 2009, 18:25

I didn't end up using BRG's SHA-1 code. There wasn't any point, really. BRG's AES code, however, is very fast (it is used in WinZip for their ZIP encryption).

I ended up using OpenSSL's SHA in trap_Sha. trap_helper has code to look up SHA contexts keyed on the destination address, so I believe it now should handle interleaved calls.

You can revert my changes to trap_Aes by commenting USE_GLADMAN_AES at the top of trap.c. trap_Sha has been modified to make use of trap_helper, in changesets 197 and 198.

**EDIT** Part of the reason I was looking at BRG's SHA-1 code was so that I could remove the dependency on OpenSSL. However, I wasn't able to find a suitable replacement for its ECDSA implementation so I didn't bother using BRG's SHA-1 code.

I've been reading over the traps and am trying to improve them where the comments say they need improvement.

Rupan

8th March 2009, 00:55

Today I was attempting to validate the current changeset with libbluray. I wanted to make sure that my changes to the source code weren't causing breakage. However, it appears that libbluray has been broken for sime time before I started working on it.

I'm testing with a region A release of "Live Free or Die Hard". I specifically remember this working a while back, so I was surprised when libbluray failed with it today. I went back through the svn revisions, bisecting from revision 168 (which I remember it working with). It turns out that svn revision 170 was what broke it.

With svn rev 169 it works fine, creating a conv_tab.bin as expected. However, with svn revision 170 it no longer works:

./convtab -d /mnt/cdrom -I <VID> -s conv_tab.bin
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 00001000, event 1)
[dlx] BREAK! PC=000010B4. WD=00000000 (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=00000000 (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 000010B4, event 1)
^C

svn info claims that the last revision to libbluray prior to 169 was by Accident, in revision 165. Revision 170 was also committed by Accident, so he/she might be the right person to debug this.

loo3aem3ON

11th March 2009, 18:27

I'm having problems building libbluray:
$ automake
configure.ac:208: required file `win32/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/libbluray/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/convtab/Makefile.in' not found
Please advise. There is still no "autogen.sh".

Rupan

12th March 2009, 01:03

run this to rebuild all of the autotools cruft in one shot:

# autoreconf -sfi

The win32 stuff was apparently added by someone with the intent of making libbluray work with Windows. It has been there since before I first saw the source code, so I'm assuming that the effort is abandoned. Personally I am not willing to touch the Windows application layer; the idea of writing source code for Windows is distasteful to me in the extreme.

If you're okay with it, just remove all references to win32 from Makefile.am and configure.ac then commit the changes. Otherwise just modify them locally before running autoreconf.

**EDIT** I do believe that I've forgotten to add the trap_helper source files to Makefile.am. You can add them in if you get to it first; I'll fix it later tonight when I get home.

I'm having problems building libbluray:
$ automake
configure.ac:208: required file `win32/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/libbluray/Makefile.in' not found
configure.ac:208: required file `win32/libbluray/convtab/Makefile.in' not found
Please advise. There is still no "autogen.sh".

loo3aem3ON

14th March 2009, 20:09

@Rupan: I still can't compile libbluray because of missing symbols so i'll revert your changes. You may commit a complete patch later.

@Accident: Could you please send me the AACS source code you have?

Rupan

15th March 2009, 02:26

I've committed a fixed patch in revision 201.

**EDIT**

@Accident: can you take a look at what happened between rev 165 and 170 to libbluray? I just confirmed, convtab fails for both "The Simpsons Movie" and "Live Free or Die Hard" for svn >= 170 (165 works fine).

@loo3aem3ON: svn rev 201 compilable for you?

Accident

17th March 2009, 08:08

Apologies for my tardiness. I was ready to start coding, when things at work went belly-up, only caught up on sleep now. I will try to catch up:

@Rupan: I wrote my code to be portable, so I did the Windows massaging (with a patch from a contributor to remove warnings). The Makefile.am for Windows are just stubs to tell autoconf to include required files when you do "make dist-gzip". Both Unix and Windows is just untar, and make.

When you check it out of svn, you should run "autoreconf", sometimes if the .ac files have changed, you may need "autoreconf --install --force".

@loo3aem3ON: The best I have found (sent to me by contributors) are:
101603 Dec 23 21:42 libaacs_v0.5.tar.bz2

All my attempts to reach the original author has failed. We might as well consider checking it into our repository, and start working on API to use both libraries.

Edit:
It appears that I changed segment_NextSegment() to instead of returning the "table index" to return the "table ID" of the table index. (Which I think is more correct... anyone?) but segment_SetSegment() still took "table index". So, I returned table = 60 as the next one to decrypt, and posted events for key 60,1. But when I went to decrypt it, "60 > numTables of 43". Can I get it confirmed that the BD+ code deals with TableID when you post the playback event to retrieve the keys, or should it be pure conv_tab index?

Edit2:

loo3aem3ON: In previous testing of older disks, the table index (subtable number) in conv_tab, and TableID were always the same. 0 to numSubTables in conv_tab. But in testing LiveAndLetDie, table index 34 (the first we need to decrypt) has table ID 60. We then post events to get the key, do we send 34, or 60. Looking at debugger, it appears that it uses
segmentID[0] = subtables[i].subtable_id; //(conv_tab_main.java:79)
which would confirm it uses only tableIDs when communicating with content code, whereas libbluray previously never used tableID at all. It would make sense to me that content-code uses tableID, and I should convert to subtable index internally, but it would be nice to be confident about this change :)

Rupan: So you say you can't test rev 202, but it is still broken.... what?

Rupan

17th March 2009, 10:09

I'm not in any state at the moment to analyze rev 202 (seeing as it is 2 am), but I'm still getting the same results:

$ ./convtab -I <VID> -s conv_tab.bin -d /mnt/cdrom
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 00001000, event 1)
[dlx] BREAK! PC=000010B4. WD=00000000 (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=00000000 (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 000010B4, event 1)
[dlx] BREAK! PC=000010B4. WD=FFFFFFFF (old R28 000010B4, event 1)

And yeah, might as well commit libaacs..

loo3aem3ON

17th March 2009, 19:16

I suggest to merge the code from libbluray into libaacs and call the resulting library libbluray. The current libbluray code needs to be restructured a bit. For example the dlx.c is overloaded with functions which have nothing to to with the DLX processor (eg. traces/snapshots, slots, conversion table, events, trap calls,...). It should be better to split the code into modules (e.g. DLX, trap_interface, trap_implementation, events, conversion_table, debugging_helpers).
Furthermore I would like if you could use the parameter checks without modifications from the documentation.

It appears that I changed segment_NextSegment() to instead of returning the "table index" to return the "table ID" of the table index. (Which I think is more correct... anyone?) but segment_SetSegment() still took "table index".
By "table" you mean the subtable associated with a specific *.m2ts file?

So, I returned table = 60 as the next one to decrypt, and posted events for key 60,1. But when I went to decrypt it, "60 > numTables of 43". Can I get it confirmed that the BD+ code deals with TableID when you post the playback event to retrieve the keys, or should it be pure conv_tab index?
I don't fully understand what you mean. The subtable ID can be higher than the total number of subtables. I don't know which part of the player generates the events 0x0110 and 0x0220. It's still unknown what exactly the second parameter of EVENT_#0110 means. Somehow it's related to the number of the m2ts file. The second parameter of EVENT_#0220 seems to be the subtable ID == number of m2ts file.

Edit: It's known that EVENT#0210 is related to BD-J and since EVENT#0220 is in the same group it's probably related as well.

Accident

24th March 2009, 08:17

This is the latest version that people managed to find me me. :thanks:
libaacs_v0.5.tar.bz2 (http://uploaded.to/?id=nndy0g)

I was initially just going to check it into libbluray as a sub-directory, but it does have the two keys embedded so I thought I should wait an see first. We should probably take those keys from command-line arguments, or env vars. Or at the very least obfuscate them so the repositories do not need to worry.

Switching to bazaar has been a hassle so far, but I should have what I need in a bit..

loo3aem3ON

24th March 2009, 21:41

Have you made any progress "connecting" the BD-J platform to libbluray? Do you need help?

bumbo

24th March 2009, 23:40

This is the latest version that people managed to find me me. :thanks:
libaacs_v0.5.tar.bz2 (http://uploaded.to/?id=nndy0g)

It seems I have a more recent version of libaacs, checked out from decrypthd svn repository quite some time ago...

libaacs-svn-r51.tar.bz2 (http://uploaded.to/?id=vuu6ka)

It does not compile here because of some missing include directives. I've attached a patch to solve that.
It also requires CMake.

loo3aem3ON

28th March 2009, 00:24

Thanks for the AACS source.

I'd like to make a suggestion regarding the connection between libbluray and the experimental bd-j platform i proposed. As i understand there has been little progress and it seems to be quite a difficult task. How about running both libbluray and bdj in two different processes and let them communicate via network or some other form of inter process communication? This would probably be easier and require fewer changes to both programs.

:helpful:

SamNZDat

28th March 2009, 23:53

Have we looked into comparing the signals sent to CRT monitor(s) with signals sent to LCD or plasma monitors, or into comparing signals sent by optic fiber(s) to signals sent by TV transmitter, satellite, cable, or DSL? Is there another way to back-engineer the technology? I have a fiber optics installer's field guide, if it can be of use to anyone.

SamNZDat

29th March 2009, 03:21

Have we looked into...

Another thought or two came to me in the last couple of hours -- just trying to think outside the box here -- how about two or more Daemon Tools mounts of Blu-Ray images, identically timed and of identical (at least as far as humanly readable) content but differently coded, running in parallel, perhaps a way to ferret out watermark? Or running rsynced Blu-Ray images by FTP -- again, to ferret out watermark, watch for decryption clues, or? Or comparing Blu-Ray behavior with the behavior of ray trace programs, for decryption clues or other objects?

Rupan

30th March 2009, 03:19

This thread specifically relates to BD+ copy protection, so please keep the discussion on-topic.

There is no need to transmit all 50GB of video data over the internet. It would be trivial to write an application that produces a hash table of small video segments and only send the differing pieces.

krosswindz

31st March 2009, 06:33

This might seem to be a stupid question, I just checked out the bdplus from Assembla. I went through the source tree seems like the configure script is missing for libbluray. Are there any instructions on building libbluray?

Accident

31st March 2009, 06:35

"autoreconf", or if needed "autoreconf --force --install". Traditionally you don't check in files that are generated when building (like .o files etc).

But perhaps I should put that in the README.txt

krosswindz

31st March 2009, 14:38

I did autoreconf but it failed due to missing files. where can I find these files? Tried going through all the 36 pages in this thread, not sure which post may point to it :(.

configure.ac:3: required file `build-aux/install-sh' not found
configure.ac:3: `automake --add-missing' can install `install-sh'
configure.ac:3: required file `build-aux/missing' not found
configure.ac:3: `automake --add-missing' can install `missing'
src/Makefile.am: required file `build-aux/depcomp' not found
src/Makefile.am: `automake --add-missing' can install `depcomp'
autoreconf: automake failed with exit status: 1

KenD00

31st March 2009, 15:21

configure.ac:3: `automake --add-missing' can install `install-sh'

Have you tried this?

:rolleyes:

krosswindz

31st March 2009, 16:01

Have you tried this?

:rolleyes:

Thanks, I guess I shouldnt try building things when I wake up first thing in the morning :readguid:

kkloster21

1st April 2009, 03:05

i don't want this to come off the wrong way, but if anybody needs a tester for the BD+ implementation in dumphd, with respect to the new MKBv9 and MKBv10 discs, i have several of these discs that i've already tried. The MKBv9 discs still produce a conv_tab.bin file that corrects some of the broken video (but not all of it) and the MKBv10 discs produce an error.

again, i have more info if anyone is interested.

Accident

1st April 2009, 04:42

Really, you should try the latest tarball, unless you have reason to want to try the latest version in repository, which is this case would be valid since we are working on the new changes... :)

loo3aem3ON

1st April 2009, 18:04

which is this case would be valid since we are working on the new changes... :)
I'm regularly checking my private message if you have any questions. Probably i could do this last step (connecting BD-J platform to TRAP_ApplicationLayer) in 1 week so everyone else shouldn't need more than 2 months. :)

dirio49

16th April 2009, 12:52

Any news? Just curious?
Progress?
Thanks

loo3aem3ON

18th April 2009, 01:05

Any news? Just curious?
Progress?
Thanks
According to Accident the latest revision (203) from April 3rd is supposed to have the connection between BD-J platform and BDVM platform. Accident can't test it though because all the computer he/she has access to are running osx and not linux (i386 compatible).
KenD00 has started working on this as well.
I'm busy at the moment and i doubt i will have much free time in the near future.

Rupan

23rd April 2009, 07:01

Sorry for the long wait. I've had other things keeping me busy.

I've taken a look at the java BDJ jar that Accident sent me. Of specific interest are two classes:

org.bluray.system.RegisterAccess
com.philips.bluray.system.RegisterAccess

Although they do not reveal anything we do not already know about the player status registers, they do expose some of the binary API in the native libraries. They also confirm the functions of several PSRs.

103 = PSR_APPL_TO_BDPLUS_DATA
104 = PSR_BDPLUS_SHARED_DATA

Some subroutines to target may be:
private static native int nGetGPR(int i);
private static native void nSetGPR(int i, int j);
private static native void nSetPSR(int i, int j);
private static native int nGetPSR(int i);
private static native String nGetPreference(int i);
private static native void nSetPreference(int i, String s);

which are all defined in libbdj.so. I've done a small bit of MIPS assembly, but I'm by no means proficient with it. I'll give them a shot...

One more thing. There is just too much code in the BDJ archive for one person to recreate. Anything other than simple classes requires a significant amount of work to fully recover, and the native functions are going to be a real bear. I need some help from other developers to decide what I should target for reversing; let me reiterate that I simply cannot reverse everything.

Rather than reverse engineer the binary MIPS libraries I suggest that we use them directly. We can write a shim in C and cross-compile it for little-endian MIPS, then run it using qemu-mipsel. IPC can be performed using named pipes or sockets. I realize that this is a completely separate track from the BDJ implementation on Launchpad, but it might be simpler in the short term and it offers the advantage of automatically supporting new versions of the "official" binary-only libbdj copied from player firmware. Comments?

Accident

27th April 2009, 01:34

Sorry for the delay, busy here too.

I can reverse engineer MIPS quite well, but small functions only, anything larger is not really realistic.. I'm not a fan of Java, as to me that cuts out too many platforms (I know, it is supposed to be more portable). I've even toyed with the idea of simply recording PSR interactions for each movie, but that's hardly a great idea. IPC is done with Unix sockets on this platform afaik, and they directly use /proc/mem for DiscoveryRAM.

dirio49

31st May 2009, 22:52

Hello, Me again

Just curious as to how is it going?
any progress?

thanks

Wombler

1st June 2009, 08:30

Funny, I wondered that myself, as it's the first of June now and there hasn't been a post since April in what was previously a very active thread.

Hopefully things are still progressing.

Wombler

kreet

28th August 2009, 06:29

im thinking about picking up this project to try and get support for the latest titles. are any of the original devs still around?

Accident

28th August 2009, 06:36

I am. Just very busy with many projects. The latest sources has both aacs and bd+ sources, which needs to be merged, or at least have a top level tool that links with both to demonstrate full functionality.

There were still some compatibility issues with latest keys, but not enough to put anyone off working on the project.

kreet

28th August 2009, 09:09

Accident:
Ive grabbed the code and will start working on it. do you have the snapshot packages?
I think another big thing is that we dont have working processing keys for > v10 (in aacskeys). we need to get these keys in order to be able to work on newer titles.

arzenami: are you or anyone else working on this?

KenD00

28th August 2009, 19:04

The latest sources has both aacs and bd+ sources, which needs to be merged

This has rather low priority, much more important is the fact that the latest BD+ versions interact with the BD-J layer, this should be implemented first before messing up with stuff that works. And there is still much work to do on the BD-J machine itself.

I think another big thing is that we dont have working processing keys for > v10 (in aacskeys). we need to get these keys in order to be able to work on newer titles.

No, i don't think that this is necessary. AFAIK BD+ can use the Volume ID off the disc but nothing more AACS releated stuff (maybe this can change in the future?), for discs we don't have a Processing Key you can still check if the Conversion Table seems to be valid, without actually watching if the movie runs without errors.

:rolleyes:

880

29th August 2009, 00:38

I think another big thing is that we dont have working processing keys for > v10 (in aacskeys). we need to get these keys in order to be able to work on newer titles.

Try working on the older (i.e. easier ?) titles first. 28 Days Later, Robocop, Taken, The Happening, etc.

These are all discs we have the keys for, but also have BD+.

kreet

30th August 2009, 07:15

ok i think ill focus on getting it to work on some older titles first. need to go out and buy a few :P

Accident

30th August 2009, 07:23

Ah yes BD-J. Guess I lost interest when it forces the use of java :) Can we do/fake/get-around BD-J and Java somehow? Re-implementing, or emulating response needed? Although it could potentially send any data to BD-J, which then is entirely open to any implementational cipher or similar, before replying back. This would make things difficult.

Although, it would not be as easy to hide anything too clever in java.

Do we have better understanding of how it is currently used? Just random data, or does it perhaps do "simpler" checks, like if we are to play title 3, it should have title 3 set in the BD-J reply.

Wombler

30th August 2009, 10:45

Great to see some activity here again as you guys are the only ones with the skills to be able to attempt this.

Wombler

kreet

30th August 2009, 12:52

Accident: i think we need to spend time reversing a working player to get a better understanding of the way the BD-J interaction works.

are you on irc somewhere or do you prefer pms?

kreet

31st August 2009, 12:53

is there a list of titles which we know use the BD-J stuff? im about to buy some discs to work with and want to make sure i get the most useful ones

Revgen

31st August 2009, 22:14

I believe the Pirates of the Carribean movies do. The latest one released on BD is Pirates of the Carribean: At World's End.

setarip_old

31st August 2009, 23:31

These 3 releases contain/utilize BD-J:

Behind Enemy Lines
The League of Extraordinary Gentlemen
Speed

The following is from this link:

http://74.125.95.132/search?q=cache:YRB0Qfh6s7wJ:netblender.com/main/resources/wikipapers/bd-j-for-independent-blu-ray-production/+BluRay%3Dtitles%3DBDJ&cd=7&hl=en&ct=clnk&gl=us

Notably, all Blu-ray titles from Universal Studios have a feature called “U-Control” which is essentially a series of in-move extras that can be activated by the viewer at any time during the movie with their remote control. Commentaries, games, picture-in-picture storyboards and production still, and other such interactive features are accessed via U-Control. U-Control is made possible by BD-J.

All Blu-ray titles from Fox are authored with BD-J pop-up menus, similar in function to how DoStudio creates pop-up menus.

Paramount, Disney, Sony and Warner are all making heavy use of BD-J menus and features (The Dark Knight, anyone?). And any disc that advertises BD-Live or any type of iPhone integration (such as NetBlender’s BD Touch) – that’s BD-J, too. Universal’s Fast & Furious is the first major studio Blu-ray title to make use of the iPhone-Blu-ray interaction.

In fact, any time the trade press takes the time to review an innovative feature on a Blu-ray disc, the feature they are referring to is enabled by BD-J. A quick perusal of Blu-ray Disc Reporter (http://www.blu-ray-disc-reporter.com) reveals numerous such reviews. Here’s one about an interesting feature on Sony’s Nick and Norah:

Comedy Nick & Norah’s Infinite Playlist comes to BD with a special feature that perfectly suits the film’s musical heritage. Nick & Norah’s Interactive Playlist gives viewers background information and trivia on the songs and artists featured in the film, and the option to create custom playlists which can be shared with friends by email.

KenD00

1st September 2009, 04:47

Well, i haven't looked further into this, but i would be careful by assuming that every title that uses BD-J uses a BD+ version that interacts with the BD-J layer. Maybe this is true for recent titles but maybe not for older ones.

In any case i know that BD+ on both X-Files movies use the BD-J layer.

:rolleyes:

kreet

1st September 2009, 07:57

KenD00: which region are those discs from?

setarip_old: are those title BD+ and are they v10 or lower?

kreet

1st September 2009, 12:16

I believe the Pirates of the Carribean movies do. The latest one released on BD is Pirates of the Carribean: At World's End.
these movies are Disney not Fox so no BD+

[Color=Blue]These 3 releases contain/utilize BD-J:

Behind Enemy Lines
The League of Extraordinary Gentlemen
Speed

do you know if any of these cause problems for libbluray/bdvmdbg?

setarip_old

1st September 2009, 20:01

@kreet

I merely provided you with information I located on the 'Net in response to your request for titles that have BD-J. I have no personal knowledge regarding them.

Rupan

2nd September 2009, 02:11

Two titles to try are "The Simpsons Movie" and "Live Free or Die Hard". As I remember, one or both of these movies is broken with libbluray. They both work with BDVMDBG (Java) and used to work with libbluray, but something was changed in libbluray that broke them. I think the break happened after r165. Neither uses the BD-J link though.

As far as reversing the Blu-Ray VM, there is a lot of code. A lot. The MIPS linux shared library I was given was about 350 kilobytes and the JAR is quite large as well. There is far too much code for one person to reverse; if it is to be attempted it will require a team of several experienced MIPS reverse engineers. There are a lot of smart people here on the forum, so it might not be unreasonable to attempt this undertaking. I've done my share of reversing i386 code but am not familiar with the MIPS instruction set.

There is another possibility that I've been thinking about for a while. It should be possible to start a fully virtualized MIPS system using QEMU and run the unmodified binaries in it. I would be extremely hesitant to run that code on bare metal (with qemu-user) since it is trivially easy to write a program that could detect when something isn't right and overwrite every partition table in your system as well as possibly damage hardware. The problem is exacerbated when you consider that the code, if it doesn't already, will probably require root privileges.

If there is interest from others I am willing to help, but I cannot dedicate more than 4 or 5 hours per month to this project (as much as I am interested in it). Let me know what I can do to help you.

Accident

2nd September 2009, 06:46

Some people asked for the BD+ binary to be shared again, the files I have is about 1.2GB in size. Any free/anonymous uploading places that can take 1.2? Otherwise I will split it into parts.

@rupan: I did not realise I broke diehard in newer, I was under the impression that I fixed it. This I can take a look at immediately.

kreet

2nd September 2009, 08:37

As far as reversing the Blu-Ray VM, there is a lot of code. A lot. The MIPS linux shared library I was given was about 350 kilobytes and the JAR is quite large as well. There is far too much code for one person to reverse; if it is to be attempted it will require a team of several experienced MIPS reverse engineers. There are a lot of smart people here on the forum, so it might not be unreasonable to attempt this undertaking. I've done my share of reversing i386 code but am not familiar with the MIPS instruction set.

i have quite a bit of experience reversing mips. even still 350k is a hell of a lot. but hopefully we should be able to use some standard vm (like the cvm) and just make changes where things fail. can someone pm me that mips bin and the jar?

There is another possibility that I've been thinking about for a while. It should be possible to start a fully virtualized MIPS system using QEMU and run the unmodified binaries in it. I would be extremely hesitant to run that code on bare metal (with qemu-user) since it is trivially easy to write a program that could detect when something isn't right and overwrite every partition table in your system as well as possibly damage hardware. The problem is exacerbated when you consider that the code, if it doesn't already, will probably require root privileges.

in my experience its relatively easy to tell that you are running on top of qemu. i doubt that macrovision/studios will allow themselves to do actual HARM tho :D id really like to try and get things running without emulating 'stolen' mips code.

If there is interest from others I am willing to help, but I cannot dedicate more than 4 or 5 hours per month to this project (as much as I am interested in it). Let me know what I can do to help you.
i have a bunch of time at the moment. the best thing i can ask for is if you could try and pass me whatever knowledge you have learnt already, and help when ever you can.

boza111

11th September 2009, 16:38

hey quick question
so what can be done , and what can not be done so far as far as watching copy protected blu ray movies goes?

I have a set top box that runs on linux and is capable of playing vc1 and blu ray movies that are not copy protected.
The receiver is a dreambox 8000 and the CPU type is MIPS i believe. Is there anything I can use from this forum to be able to play copy protected blurays on my dreambox?

cRTrn13

13th September 2009, 11:08

@boza111
I'm working on a patch for mplayer to allow bluray playback (at least those with AACS only atm). If you can get mplayer working on your mips box and decoding h.264 nicely, then it should be able to handle the AACS copy protection when the mplayer patch is released. What do you currently use for vc1 playback? Also what drive do you use?

boza111

13th September 2009, 17:50

@cRTrn13

Hey thanks for the quick reply, awesome that you're writing a patch for mplayer to allow the playback for bluray!
as far as i understand the vc1 and h.264 videos are played back directly by hardware by the broadcom 7400 chip. I have also asked in the community what exactly is used to play back vc1 and h.264 content, but I am pretty sure its handled by hardware.
Concerining the drive , right now I don't have a bluray drive in the box since it cant play copy protected movies. I play ripped copies directly from the internal harddisk or over the network. But it is possible to build in any slimline drive.
Are specific drives better for playing blu ray discs in linux?
Is there any chance of getting mplayer to work with the broadcom chip? and if not , is there still hope to play back copy protected bluray discs?

Guest

13th September 2009, 19:44

Guys, this is OT. Please open a new thread for this Broadcom/mplayer/VC-1 discussion. This thread is for discussion of BD+.

XYZ123

24th September 2009, 18:26

Anyone working on BD+ at the moment?

cRTrn13

24th September 2009, 19:36

Anyone working on BD+ at the moment?

Yup. Someone is.

boza111

30th September 2009, 16:29

hey cRTrn13 how is your tool coming along :) i found your blog , looks like your making good promising progress :)

cRTrn13

1st October 2009, 10:38

hey cRTrn13 how is your tool coming along :) i found your blog , looks like your making good promising progress :)

Lol - haven't updated that a while - maybe i'll post an update. Yes, it's fairly stable for AACS - just working on some little bits and pieces.. :) It's coming along nicely though! :)

kreet

4th October 2009, 10:10

im working on reversing some players to understand the whole bd-j thing and grab new keys. theres a lot of work to do. if someone wants to volunteer to implement some of the things i discover, it would be very helpful. pm me.

kreet

6th October 2009, 10:56

i think im gonna start a new thread for bd+ reversing and updating libbluray support to current titles. what do you guys think? there is a crapload of work that needs to be done and i could really use a hand, both on implementation and reversing.

if you have leet skillz in x86 asm, java reversing, c/c++ coding and/or java coding pm me here or ping me on irc://irc.efnet.net/#doom9

GLUBSCH

6th October 2009, 12:14

Sounds great, kreet! I´ve got leet skillZ in testing stuff. ;)

kreet

6th October 2009, 14:29

new thread here: http://forum.doom9.org/showthread.php?p=1332018

let the games begin!

Accident

13th January 2010, 07:48

Any readers out there who can run the java Debugger and record snapshots still? I would like to fix more of the small differences in the C version. My old saved snapshots may no longer be current.

Edit: Counting differences I get:

10: 1 times
140: 192 times
420: 33 times
530: 350 times

I completely re-did the DiscoveryRAM, with the proper memory dumps, so it no longer differs. sha(140) differs only in the work area. SlotRead(420) differs since our flash.bin files are different from starting.

As for why Finish (010) differs one time, i do not know. Now it runs much better, and has been committed. I have had reports of old bdplus disks playing, which is encouraging.

Did we ever decide on a name for EVENT_220 ?

Edit:

Thanks Mike Chen.

Mike Chen

21st January 2010, 10:37

Did we ever decide on a name for EVENT_220 ?
ComputeSP

Accident

28th May 2010, 03:14

I know the current BD+ code we have will not handle Avatar, but for those curious this is what happens:

UDF-fs INFO UDF 0.9.9 (2005/21/07) Mounting volume 'RED_BIRD_2D_F6'
bdplus.c:105: [bdplus] loading BDSVM/00000.svm and flash.bin...
[snip]
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING reading from PSR103!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING reading from PSR104!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING writing to PSR104!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING writing to PSR102!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING reading from PSR103!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING reading from PSR104!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING writing to PSR104!
bdsvm/trap.c:983: [TRAP] TRAP_ApplicationLayer: WARNING writing to PSR102!
[snip]
bdsvm/interface.c:128: [interface] trap_ConvertionTable(000F3CCC, *002EFAC8)
bdsvm/trap.c:164: [TRAP] TRAP_FixUpTableSend(998604/3994416)
bdsvm/segment.c:50: [segment] Starting decode of conv_tab.bin: 0x1012efac8 (998604)
bdsvm/segment.c:92: [segment] num tables 193
bdsvm/segment.c:127: [segment] Table 0 ID 00000001, 1 segments
bdsvm/segment.c:206: [segment] Table done. Setting ptr to 00000010
bdsvm/segment.c:127: [segment] Table 1 ID 00000002, 442 segments
bdsvm/segment.c:157: Segment 1 offset 00000702 -> 69 entries
bdsvm/segment.c:157: Segment 2 offset 00000C6A -> 103 entries
bdsvm/segment.c:157: Segment 3 offset 0000147A -> 118 entries
[snip]

Now I would suspect it will fail after this, when it attempts to decode the segment keys, as the (missing) BDJ interface responds incorrectly. But since it was an hiccup for Sly-soft as well, I presume there will be an upcoming road-block that I can not yet see?

Edit:

dirio49: that is fascinating for sure. Nice to have it laid out. I did notice that it requires the VM to run permanently, but don't know anything about mk.enc.

dirio49

29th May 2010, 13:23

@Accident

i don't know if you have seen this, fengtao posted it over at dvdfab forum (http://forum.dvdfab.com/showthread.php?t=5175).
Hope it helps

BTW it that the newer BDVM that people are working on videolan?
later
Generation 1
- started with "The Day After Tomorrow" on 08jun2007
- Cryptography Research, Inc. was making BD+ code back then.
- some FUEs (FixUp Element) were watermarked

Generation 2
- started with "Mrs Doubtfire" on 20dec2007 and not with Hitman as previously believed
- TRAP_MediaCheck() is used to check if the disc is original

Generation 3
- started with "Jumper" on 14apr2008
- Macrovision, Corp. bought the technology from CRI.
- useless FUEs were produced; when patching the movie, these have to be skipped
- TRAP_DebugLog() is used to print messages from BD+ code
- starting with the James Bonds movies a BDJ/BD+ Handshake is used

Generation 4
- starts with "Futurama: Benders Game" on 22aug2008
- Handshake becomes more complicated; the BDJ code, after exchanging data with BD+,
gets to decrypt and run a hidden Java object that does further checks (mk.enc)

Generation 5
- starts with "Slumdog Millionaire" on 02feb2009
- Handshake return codes are not standard anymore (1000=ok, 200X=error card, etc);
they are now different for each movie
- MKJ.enc fallback package disappears, MK.enc will do the checks

Generation 6
- starts with "Valkyrie" on 31mar2009
- MK.enc is moved from 77771.jar to 77773.jar

Generation 7
- starts with "Ice Age 3" on 18aug2009
- new 00003.svm appears; it was mentioned in the patents, but not used till now

Generation 8
- starts with "Avatar" on 06mar2010
- Macrovision Corp. gets rebranded to ROVI Corp.
- although still called Handshake, the BDJ/BD+ exchange becomes permanent, not just at the start of playback.
- the use of Native code begins; this adds support for new TRAPs on-the-fly

ro-ee

16th February 2012, 23:26

It's now almost two years later. Any news on BD-J or BD+?

Accident

17th February 2012, 04:33

I do believe VLC-2.0 RC is available which should include some of the bluray code. I don't know specifically which parts though. If sources are not available there, I guess we could check the latest known version into github or something.

ro-ee

17th February 2012, 10:16

Is there any point in providing only the *.svm files for debugging/testing purposes. Or does one need also VUK/VID/??? in order to have the VM run correctly? I reckon the Java files would also be needed for that.
I was just toying with the idea that the bd+-lib could calculate the convtable once and for all, so it wouldn't have to do it any time the disc is played, the files would simply be cached on the file system, just as the VUKs already are.

monk3y

22nd February 2012, 13:08

@Accident

VLC 2.0 includes libbluray, but not libaacs.

ro-ee

22nd February 2012, 17:01

@Accident

VLC 2.0 includes libbluray, but not libaacs.

libbluray loads libaacs (and libbd+) if they are found, this means if you have them both (and the keys), then VLC should be able to play also encrypted BDs.

What I was asking earlier was the BD+ thing, which apparently needs some help from BD-J. Don't know if there is some support for it in VLC, and I also don't know how these two will/should/can work together.

Perhaps someone in the know can explain to me how the current BD+ situation is. From my knowledge, the TRAP instructions are old news, next up is BD-J interaction (also through Traps?), and recently I can upon someone claiming that X86 DLLs are loaded by the BD+ code. How does this even work on a standalone player?

HWK

11th March 2012, 02:01

Perhaps someone in the know can explain to me how the current BD+ situation is. From my knowledge, the TRAP instructions are old news, next up is BD-J interaction (also through Traps?), and recently I can upon someone claiming that X86 DLLs are loaded by the BD+ code. How does this even work on a standalone player?

I think for standalone it doesn't. It is limited to pc only.

ro-ee

12th March 2012, 14:04

I think for standalone it doesn't. It is limited to pc only.

So there must be a way in the code to detect if it runs on a certain player and branch out to different methods. I thought this was ruled out a while ago? Or has this been the case only recently?

HWK

12th March 2012, 18:33

All companies who are planning to license their player are required to provide blue print of their player to Rovi formerly macrovision.

Let say all standalone player has blue print of ABC and pc based player have 123. With this in mind BD+ code could perform initial check and they may have added logic if it is 123 run additional code on top of it.

To make matter worse player would be happy to run the code when player are running and playing movie. As such there is no need for autorun or anything along that line. Even admin account is not required to run code and while the code is running it can sniff around on pc just like trojan would do.

With this mind code is free to do whatever it wants.

xabaras414n

12th May 2020, 18:15

Greetings,

I'm looking into packaging the "bdplus-vm0" files for my Linux distribution (Slackware).
May I ask if a "stable" source for that archive or its files exist and if there any issues with redistributing those files?

candela

12th May 2020, 18:43

Greetings,

I'm looking into packaging the "bdplus-vm0" files for my Linux distribution (Slackware).
May I ask if a "stable" source for that archive or its files exist and if there any issues with redistributing those files?

these files contains keys and memory areas of Cyberlink PowerDVD so I don't think you can include them :)

xabaras414n

12th May 2020, 20:40

Ok thanks, I guess that also explains the PowerDVD-specific instructions that are shown on currently-unsupported BD-J discs...

binaryflow

10th June 2023, 09:30

Despite "no worth the effort"...

Any interest on thia these days for the sake of science and such?

RealSnoopyDog

14th June 2023, 01:23

You only joined this forum just to spam the threads with meaningless crap @binaryflow ?