Finally handling BD+ (?) [Archive] - Page 3

Disabled

9th September 2008, 22:07

I'll just quote Oopho2ei:
I have setup and account now. If you want to join our team please send me a private message for details. I will choose who gets access to the repository.

And about the AES implementation:

Leave Trap_AES for now. I haven't even described it yet. :rolleyes:

Thus: Everyone who wants to help, just write him a message about the details. And we'll concentrate on the easy traps for a start until TRAP_AES and others are documented.

The next feature I implemented was an .svm parser. Now you can load a segment of an svm file, the BDSVM_CC header is verified, the length is calculated and then the data is copied to mem and the first 0x1000 bytes are zeroed.
And I again rewrote the dasm part, as It was a horror to debug both earlier versions. I hope thats the last time I'll do that.

Next stop: implementing traps
*edit* perhaps I'll restructure the code first...
*edit2* oh and perhaps I'll also rewrite the debug trace functions first...
restructuring the code is also a good thing, so more ppl can work on the code without the need of backporting or merging code.

Oopho2ei

9th September 2008, 23:47

Thus: Everyone who wants to help, just write him a message about the details. And we'll concentrate on the easy traps for a start until TRAP_AES and others are documented.
I don't hide the details. If i knew i would post them (except player specific stuff or course). Of course you can ask me and depending on your question(s) i will take a look.

The next feature I implemented was an .svm parser. Now you can load a segment of an svm file, the BDSVM_CC header is verified, the length is calculated and then the data is copied to mem and the first 0x1000 bytes are zeroed.
And I again rewrote the dasm part, as It was a horror to debug both earlier versions. I hope thats the last time I'll do that.
You are doing well. It's good to have you in the team. :)

For those who want to play around a bit with the emulator here is an extended program counter trace: http://uploaded.to/?id=gx84gh (~4 million entries)
If you need other (extended) traces please leave a note here or send me a private message.

Thanks to all who made suggestions where to host our software development project. We currently have a working repository but this is still very helpful if we get trouble and have to move to another location.

KJI

10th September 2008, 00:08

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.

Oopho2ei

10th September 2008, 16:22

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.
The code looks very clean and well structured. I like it but i suggest to convert the image (section from 00000.svm) from big to little endian before execution. This should be more efficient compared to converting the data/code on every access.

Btw. if you want to test a trap implementation use the pc trace from posting #110 and ask me for some memory snapshots. You can also send me a modified memory snapshot so you can run your own program with your own data on a fully implemented player. I would need the initial register values and where to start and stop execution. But please comment your program and all the other changes you made so i can verify that no player specific data is retrieved. You would get the result as a memory/register file snapshot.

loric

10th September 2008, 18:16

The code looks very clean and well structured. I like it but i suggest to convert the image (section from 00000.svm) from big to little endian before execution. This should be more efficient compared to converting the data/code on every access.

I beg to differ. In case of self-modifying code (and in the future bd+ may become that), what you suggest would break things. My suggestion is to behave as close as possible as the real BD+VM which, AFAIR, doesn't do any endianess conversion at load time.

BTW, great job.

Oopho2ei

10th September 2008, 19:39

I beg to differ. In case of self-modifying code (and in the future bd+ may become that), what you suggest would break things. My suggestion is to behave as close as possible as the real BD+VM which, AFAIR, doesn't do any endianess conversion at load time.
Please elaborate on this seĺf modifying code problem you see. If all memory accesses of instructions and traps are adapted according to our memory format (e.g. big endian or little endian) it's impossible for the program running on the vm to tell any difference. We could even store our instructions in some weird looking sql database tables. As long as everything is correctly implemented our memory organisation is fully transparent to the program which is running on our emulator. But what does change is the performance of our emulator.

Edit: The following archive contains a series of 16 snapshots taken after each of the first 16 trap executions. Example: post_trap_mem_000.bin + post_trap_reg_000.bin is a snapshot after the execution of the first trap which is Trap_AES (TRAP #110). Each snapshot is supposed to be loaded by your emulators with every new trap call you encounter when executing the given program (see the program counter trace in posting #110). In other words instead of executing the trap you only load the corresponding snapshot. After the 11th trap call you will find yourself right in the middle of the decryption process of the second section from the second file (00001.svm). Also there seem to be pictures in the *.svm files somewhere. So it's getting more interesting soon. :)

post trap snapshots: obsolete. see posting #130

(for those who don't know: we are executing the first section of http://uploaded.to/?id=jqstc8 which is one of the files stored in the BDSVM directory of every BD+ protected blue ray disc)

sakman

11th September 2008, 09:42

Looks like a date is also part of the header:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4

07D7h=2007
0Bh=11 (November)
06h=6

bmnot

11th September 2008, 10:11

ok... sorry for the delay.

It's interesting how things evolved (nice work!).

As announced in my last post, I implemented a Java BDVM together with a graphical debugger.
It executes the BD+ program correctly until the 0x520 trap (memory dumps and registers match) - just as the other two VMs do, which I used as an orientation/prototype. The code is pretty similar (in my notion, it is slightly cleaner and more compact than my C implementation due to Java's convenient ByteBuffer classes which take care of endianness; Apart from that, the graphical Debugger is actually less code than the VM :) (294 SLOC vs. 245 SLOC) ; the implementation is big endian throughout; I just started looking at the traps).
I don't know whether this will be useful. I mainly implemented this for fun and educational purposes (for me and maybe others).

I agree that the implementation by KJI's friend is very clean (this probably doesn't matter, but I think 'vm->watchdog-=2;' in 'void ExecuteInstruction(BDVM *vm)' is not correct - see post #25).

Below is a screenshot of the debugger (ok, no screenshot... no idea where to upload it to.. it's included in the zip). BTW, the disassembling is dynamic (updated after every step).

Source (2 Java files - 1 VM and 1 Debugger) is available from here: http://uploaded.to/?id=s46hrd

derbeDeus

11th September 2008, 11:24

Looks like a date is also part of the header:
42 44 53 56 4D 5F 43 43 01 00 00 00 01 07 D7 0B 06 00 00 00 00 07 EC E4

07D7h=2007
0Bh=11 (November)
06h=6

The BDSVM block format is:
BDSVM_header
-----8<------
magic 8 bytes - is string 'BDSVM_CC'
?version 1 byte - always 1?
?unk9 4 bytes - usualy 1 (sometimes 0x2000 in AVP2, Jumper)
year 2 bytes
month 1 byte
day 6 bits
chunkSize 26 bits - always 0? (don't remember why I named it)
datalen 4 bytes - size of code chunk following the header
----->8------
ContentCodeBlock of datalen bytes
CC_signature 40 bytes

that's padded with 0's to 2MB boundary

Oopho2ei

11th September 2008, 18:44

As announced in my last post, I implemented a Java BDVM together with a graphical debugger.
Looks awesome! Could you write a small howto to get it all working? My first attempts failed and i would like to try it :)
javac Debugger.java BDVM.java
Debugger.java:110: error: The method actionPerformed(ActionEvent) of type Debugger must override a superclass method
public void actionPerformed(ActionEvent e) {
(javac invokes gcj-4.3 here.)

Could you also please load a snapshot for traps which are currently not implemented? See postings #114 for details.

sakman

11th September 2008, 19:49

The BDSVM block format is:
BDSVM_header
-----8<------
magic 8 bytes - is string 'BDSVM_CC'
?version 1 byte - always 1?
?unk9 4 bytes - usualy 1 (sometimes 0x2000 in AVP2, Jumper)
year 2 bytes
month 1 byte
day 6 bits
chunkSize 26 bits - always 0? (don't remember why I named it)
datalen 4 bytes - size of code chunk following the header
----->8------
ContentCodeBlock of datalen bytes
CC_signature 40 bytes

that's padded with 0's to 2MB boundary

Thanks derbeDeus. Matches what I saw looking at several .svm files from different discs. I did notice that the 4 bytes of "?unk9" changed to 2000h with that bunch of March/April 2008 discs (Hitman, Jumper, Alvin, etc.). Interesting that those discs also broke the initial BD+ support in AnyDVDHD.

bmnot

11th September 2008, 21:57

Okay, I made things a little simpler:
The binaries, a readme and an ant build script are included in this updated version: http://uploaded.to/?id=khplo8
Java 1.5 or newer is required in order to compile and run the Debugger (not sure whether gcj will work).
Ant is helpful for compiling.

Regarding the traces...
Executing with the help of the dumps was successful :cool:
I'm not sure whether this is already clear: r1 seems to contain the return values from the trap calls.
Apart from r1, the registers were always identical to the dumps (if I haven't missed something).
Note that loading the mem dumps and register dumps is not automated in the debugger. Currently this has to be done manually.

The Traps were: (<dump no>) <trap id> <instruction counter>
(0) 110
(1) 110
(2) 520
(3) 520
(4) 520
(5) 110
(6) 110
(7) 110
(8) 540 4527
(9) 110 13536
(10) 230 13547
(11) 110 22408
(12) 230 22419
(13) 110 31289
(14) 230 31300
(15) 110 40161
((16) 230 40172)

Oopho2ei

11th September 2008, 21:59

Some notes about how to implement Trap_AES(0x001EF748, 0x001EF748, 0x00000080, 0x00016764, 0xFFF10001) and similar.

Remember the declaration: UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);

The last parameter (opOrKeyID) is 0xFFF10001 so we have to decrypt [len] 128bit blocks of [*src] using ECB mode with the 16-byte key [*key] and write the results to [*dst]. In ECB mode every 128 bit block is decrypted independently from all other blocks so i will only demonstrate the decryption steps for a single block.

Example snapshots: http://uploaded.to/?id=7sz6pu
This archive contains two snapshots. One was taken directly before the Trap_AES call we are looking at and one directly afterwards. You can use this to verify your implementation.

Let's look at address 0x001EF748 to find the first data block we have to decrypt and at address 0x00016764 we will find the key we have to use:
data: c5 ed 4c 67 e4 45 ca e9 40 e0 b7 89 3a c9 54 a6
key : e6 d3 0c b1 d4 29 4c 88 e4 66 29 06 34 35 25 be

Our memory content has been converted from big endian to little endian which breaks the decryption process using default aes implementations but we can simply revert the conversion:
data: 67 4c ed c5 e9 ca 45 e4 89 b7 e0 40 a6 54 c9 3a
key : b1 0c d3 e6 88 4c 29 d4 06 29 66 e4 be 25 35 34

We apply this to all 80h blocks (16 bytes each) we have to decrypt and then pass the result to a default aes-128-ecb implementation. In this example I simply put the first block in a file source.bin and called openssl from the command line:
openssl aes-128-ecb -d -K b10cd3e6884c29d4062966e4be253534 -iv 00000000000000000000000000000000 -in source.bin -out result.bin

The result is: 9d df 63 6e 37 cf b6 c6 dc b0 2d c3 a3 ca a7 f1
Now all that's left to do is to convert this block from big endian to little endian again and write the result to 0x001EF748. You can verify these steps by looking at the two snapshots i posted above. If the decryption completed without any errors we return 0 (return values are written into r01)

Okay, I made things a little simpler:
The binaries, a readme and an ant build script are included in this updated version:
I got it running!! :)

Some suggestions:
- for branches you better show the absolute address and the not the relative address
- could you automatically load the snapshots? :)
- add a data view window (maybe double the number of columns for the register view and cut its height by half to make space for a small data view window below)
- reset the watchdog timer with ever trap call to 0x7FFFFFFF
- display the parameters of the trap call directly before it is executed (e.g. by commenting the values in the stack window)
- support of data breakpoints
- color the registers red which have been changed since the last step/run
- maybe assign some hotkeys

Again good work! :)

derbeDeus

12th September 2008, 10:09

Thanks derbeDeus. Matches what I saw looking at several .svm files from different discs. I did notice that the 4 bytes of "?unk9" changed to 2000h with that bunch of March/April 2008 discs (Hitman, Jumper, Alvin, etc.). Interesting that those discs also broke the initial BD+ support in AnyDVDHD.

I think the change was later, with Alvin and AVP2 (as they also changed to MKBv7).
In my copy of Hitman it is 1 (!?)

As a side note, Fox has still MKBv7 for its titles, while others (Lionsgate) changed to 8 and 9. Probably they trust their BD+ code more than AACS protection. duh :p

gioowe

12th September 2008, 15:40

Maybe someone would like to implement the following trap:

UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
It's the trap number 0x8010.

It seems to just log debug information from the BD+ VM to somewhere.
txt is a pointer within VM space to a character sequence (single byte) with length len (not null terminated).

Maybe the BD+ implementations use it and we'll see them send some information :)

Oopho2ei

12th September 2008, 17:29

More and more people keep asking the same question about why a particular trap call didn't change anything in the guest system (that 4MB memory + registers you see).
The answer: The traps are executed on the host system side in which the guest system is 'embedded'. So a trap can make changes in the host system (e.g. replace some bytes in the audio or video stream) which are invisible to you because i only show you the embedded guest system for obvious reasons. So when you don't see any changes in the snapshots before and after a trap execution the trap can still have made changes in the host system.

UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);
It's the trap number 0x8010.

It seems to just log debug information from the BD+ VM to somewhere.
txt is a pointer within VM space to a character sequence (single byte) with length len (not null terminated).
This would be an example of such a trap. It changes the host system but not the guest system. Thank you gioowe for sharing this information. :)

Edit: schluppo has found a mistake in the post_trap_snapshot archive from posting #114 (post_trap_mem_002.bin and post_trap_reg_002.bin were wrong). I have recorded a second independent series of snapshots and replaced the link accordingly. Note that starting with post_trap_mem_004.bin the two series differ at address 0x4514 and 0x4518-0x451B so the third call of TRAP_DeviceDiscovery seems to introduce random data? Maybe someone could check who is writing to these locations. We need data breakpoints too, bmnot :)

schluppo

12th September 2008, 22:00

Here are some more details on the traps :

instruction #00417:

trap_110(0x1A64C,0x5550,1,0x5570,7)

returns 0x80000001

no memory / registers changed

instruction #00475:

trap_110(0x1A64C,0x5550,1,0x5560,8)

returns 0x80000001

no memory / registers changed

instruction #00839:

trap_520(1,1,0x21A0,0x1EFF84)

returns 0

writes 284/0x11C bytes to 0x21A0 (these 284 bytes seem to contain around 84 bytes header).
the last parameter holds available / written length.
second parameter is the question-id or op-id.
maximum reply length for questions / ops #1 and #2 is 0x140, for question /op #3, it's 0x1000.

instruction #02319:

trap_520(1,2,0x2358,0x1EFF84)

returns 0

writes 292/0x124 bytes to 0x2358 (these 292 bytes seem to contain around 88 bytes header).

instruction #03539:

trap_520(1,3,0x4514,0x1EFF84)

returns 0

has 4096 bytes reserved (as 0x00) at 0x4514, writes only 60/0x3C bytes to 0x4514.

instruction #04289:

trap_110(0x156DC,0x156DC,3,0x571C,4)

returns 0

writes 48 bytes to 0x156DC.

instruction #04360:

trap_110(0x17764,0x17764,2,0x16764,5)

returns 0

writes 32 bytes to 0x17764.

instruction #04435:

trap_110(0x16764,0x56B0,1,0x16764,5)

returns 0

writes 16 bytes to 0x16764.

instruction #04527:

trap_540(0x1B428,1,0,0x1EFF54,0x22060)

returns 0

this trap writes 0x74F9C bytes to 0x22060. fourth parameter adress contains number of written bytes, fifth parameter is the dst adress.

I could not find any documented trap in the patent text that would fit to trap_540.

Decryption of the result of trap_540 happens by the following sequences of trap_110 and trap_230. Blocks of 2kb are decrypted by each iteration:

instruction #13536:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #13547:

trap_230(0x22060,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x22060.

instruction #22408:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #22419:

trap_230(0x22860,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x22860.

instruction #31289:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #31300:

trap_230(0x23060,0x1EF748,0x200)

returns 0

writes 2048 (0x200 times 4) bytes to 0x23060.

instruction #40161:

trap_110(0x1EF748,0x1EF748,0x80,0x16764,0xFFF10001)

returns 0

writes 2048 (0x80 times 16) bytes to 0x1EF748.

instruction #40172:

trap_230(0x23860,0x1EF748,0x200)

returns 0?

writes 2048 (0x200 times 4) bytes to 0x23860?

This finishes the first 16 traps.

Oopho2ei

12th September 2008, 22:22

instruction #04527:

trap_540(0x1B428,1,0,0x1EFF54,0x22060)

returns 0

the snapshots indicate that this trap changes 477301 bytes of the vm memory, so i guessed that the snapshot is bad and stopped here.

I could not find any documented trap in the patent text that would fit to trap_540.
Sorry, I forgot to upload the 00001.svm. This trap (trap_540) seems to load the second section of the second file (00001.svm) at the address given by the last parameter (0x22060). The section is being decrypted later (see posting #114)

Here it is (finally): http://uploaded.to/?id=tn6kle

Thanks for reminding me. :thanks:

schluppo

12th September 2008, 23:08

The difference in your snapshot series appears since these bytes hold a timestamp.

0C09D807 2D353013 => 12-09-2008 19:48:53:45
0A09D807 550F0014 => 10-09-2008 20:00:15:85

Edit (answering to Oopho):

-i.e.
0x0C=12, 0x09=09, 0x07D8=2008
0x13=19, 0x30=48, 0x35=53, 0x2D=45
0x0A=10, 0x09=09, 0x07D8=2008
0x14=20, 0x00=00, 0x0F=15, 0x55=85

-Further, it's hard, to make out a pattern in the outputs of the calls of TRAP_DeviceDiscovery(..). In the case, that the question has an answer of max-length 0x140, the output seems to have a header of around 90 bytes containing (each word of length 32 bits, little endian, parameters are _not_ identical to - but probably depend on - the trap parameters):

<length_of_result>, 0x100010, 0x4, param2, param3, param4, ~16 times: 0x00000000, <manufacturing_date>

The ~200 bytes following the header are incomprehensible to me without any hint of interpretation. Anyway, they are the 'answer' to the 'question' asked to the device.

I'd say that more calls of this trap need to be snapshot in order to get a better base for estimations.

-More details on the traps here: http://uploaded.to/?id=xqibd0

Edit: updated text file with info on all the first 16 traps.

Oopho2ei

13th September 2008, 11:00

Thank you. Were you able to trace the key from posting #121 back to a trap result?

I'd say that more calls of this trap need to be snapshot in order to get a better base for estimations
I will try to record a long series of snapshots automatically now. :)

In general for those traps you have already implemented you should still load the corresponding snapshot for verification. If yours and mine don't match you probably made a mistake. :p I provide these snapshots primarily for you to test your implementations. But they of course also useful to study traps which are currently unknown.

schluppo

13th September 2008, 14:37

The key from post #121 is the result of this trap:

instruction #04435:

trap_110(0x16764,0x56B0,1,0x16764,5)

returns 0

writes these 16 bytes to 0x16764:

E6D30CB1 D4294C88 E4662906 343525BE

trap_110/TRAP_Aes is called three times in a row, first returning 48, then 32 and finally 16 bytes, the calls are:

trap_110(0x156DC,0x156DC,3,0x571C,4)
...
71 non-trap instructions
...
trap_110(0x17764,0x17764,2,0x16764,5)
...
75 non-trap instructions
...
trap_110(0x16764,0x56B0,1,0x16764,5)

Oopho2ei

14th September 2008, 00:20

These are the first 5000 snapshots. Each snapshot was taken automatically after every trap execution. I have made multiple independent packages you can download.

Warning: You may not mix snapshots from different series therefor i have removed the link to the old snapshot series in posting #114. If you have already implemented Trap_DeviceDiscovery you have to set the timestamp according to the snapshot series you use.

Attention: All the snapshots are little endian (swapped for performance reasons) but the DLX is a big endian processor. So depending on your implementation might have to swap the byte order when loading a snapshot.

Note:: Starting with package version v1.03 we use the new diffarchive format to store the snapshots. A sample implementation can be found here (http://svn.assembla.com/svn/bdplus/trunk/diffarchive/).

current packages:
Package (v1.03) for "The Day After Tomorrow": http://uploaded.to/?id=5dwn30
Package (v1.02) for "I Robot": http://uploaded.to/?id=svpn7q

old/deprecated packages:
Package (v1.02) for "The Day After Tomorrow": http://uploaded.to/?id=g93e2g
Package (v1.01) for "The Day After Tomorrow": http://uploaded.to/?id=yodttu
Package (v1.00) for "I Robot": http://uploaded.to/?id=mej65b

Current package contents:
/
AACS/
post_trap_snapshots/
post_break_snapshots/
BDSVM/
traces/
aacskeys_output.txt
title.txt
version.txt

How to use these snapshots:
- implement a counter 'cnt' (or any other name) which is incremented after every execution of cmd39. The initial value of the counter is 0.
- load the files with filenames [snapshot_reg] and [snapshot_mem] with sprintf(snapshot_reg,"post_trap_reg_%03d.bin",cnt); sprintf(snapshot_mem,"post_trap_mem_%03d.bin",cnt); from the trap_snapshots directory
- after your implementation of the trap terminates compare your memory and register file with the snapshot you have loaded
- if your memory and register file contents doesn't match the provided snapshot then print a warning (or whatever you think is appropriate) and overwrite your register file and memory contents with the snapshot and continue execution
- use the post-break snapshots in the same way when the watchdog timer reaches zero (or becomes negative)

Benefits:
- automatic testing of your implementation on a real world example
- program execution up to the point where video/audio playback starts
- black box analysis of currently unknown traps by looking at input and output

Oopho2ei

14th September 2008, 21:38

Another mistake i made was found: cmd3A is INSTF so it sets the instruction filter value and not the watchdog timer. I have corrected this immediately in posting #27. For a short explanation of what INSTF does please see posting #44

Thanks for reporting this! :thanks:

schluppo

15th September 2008, 00:52

Here is an edited version of bmnot's quite usefull java debugger (I apologize in advance in case that anyone is offended by my bad coding style or use of bmnot's ideas):

http://uploaded.to/?id=ph017s

Things I changed:

- Treatment of parts of trap_110, treatment of trap_520.
- No treatment of actual AES functionality of trap_110 (didn't have time to implement this in java yet). No treatment of trap_540 and trap_230.
- Automatic input of "post_trap_mem_xxx" and "post_trap_reg_xxx" when untreated traps appear (Note that all the snapshots need to be located in the working dir - i.e. the dir containing the .jar - at this point).
- I added a "trap counter" to the debug-gui.
- After pressing "run", the VM will run till the 472th trap is reached (or till PC goes out of sync or breakpoint is reached).
- After that, pressing "run" will run till the next trap is reached (or till PC goes out of sync or breakpoint is reached).
- Whenever the memory is being dumped manually, a binary statistic of trap calls (trap_calls.bin) is written to the working dir.

Things to do:

- Further treatment of traps.
- Comparison of snapshots with results of treated traps.
- (Separate option to create trap-calls statistic.)
- (Separate option to select source dir for automatic snapshot input.)

Things learned:

- The debugger is quite slow on inputting PC-traces. 4,000,000 PC values take 10min+.
- The first ~2,080,000 instructions of the BD+-VM contain 477 trap calls:

239 times trap_110
234 times trap_230
003 times trap_520
001 times trap_540

- The first 477 traps do the following: 1) Setup the device 2) get some basic info on the device 3) let the device decrypt a key (taken from the .svm or from the device?) 4) load ~500kb of data from somewhere. 5) decrypt this data using the key from step 3) and also using the device's XOR-capabilities (?).
- Using the snapshots, the PC is not getting out of sync in the first 2 million instructions, indicating that the debugger's VM is quite correct (apart from the traps).

Some questions:

- What exactly does trap_540 do? I couldn't find the byte-sequences that it writes to the VM's memory anywhere in 00001.svm. Is it XORing the data from 00001.svm with the first parameter's value (0x30303030) before writing it to the VM's memory?
- Are you sure, trap_230 is pure XOR? The snapshots indicate that trap_230 is sometimes changing more than len (third parameter) times 4 byte (could also be some mistake in the snapshots / my calculations).

schluppo

15th September 2008, 04:05

The traps from the previous post are followed by:

trap #477 at ic: 2081538 is trap_530(0,0x2514,0x2000); returns 0;
writes 0x2000 bytes to 0x2514.
trap #478: trap_110(0x24FC,0x7D360,1,0x7D350,6);
writes 16 bytes to 0x24FC.
trap #479: trap_110(0x24EC,0x7D360,1,0x7D350,5);
writes 16 bytes to 0x24EC.
...
trap #484: trap_110(0x249C,0x7D360,1,0x7D350,0);
writes 16 bytes to 0x249C.
trap #485: trap_110(0x24FC,0x7D360,1,0x7D350,6);
writes 16 bytes to 0x24FC.
trap #486: trap_110(0x24EC,0x7D360,1,0x7D350,5);
writes 16 bytes to 0x24EC.
...
trap #491: trap_110(0x249C,0x7D360,1,0x7D350,0);
writes 16 bytes to 0x249C.
trap #492 at ic: 2687589 is trap_520(1,2,0x1EFA94,0x1EF890->0x190); returns 0;
identical behaviour as previous trap_520 with same params :)
trap #493 at ic: 2687599 is trap_140(0x1ef894,0x1efa94,7c,3); returns 0;
writes 0x7c bytes to 0x1ef894. this seems indeed to be trap_sha.

After trap #493, PC goes out of sync for me even when loading post_493 snapshots. The jump (BEQZ r6, 0x28) at 0x95064 (ic: 2691187) is not taken, but should be taken according to the PC-trace (r6 = 0xFF in my run).

Oopho2ei

15th September 2008, 08:04

- What exactly does trap_540 do? I couldn't find the byte-sequences that it writes to the VM's memory anywhere in 00001.svm. Is it XORing the data from 00001.svm with the first parameter's value (0x30303030) before writing it to the VM's memory?
Trap_540 is UINT32 TRAP_MediaReadFile(UINT8 *FileName, UINT32 SectionNumber, UINT32 Unknown, UINT32 *len, UINT8 *dst);

It reads [len] bytes(?) of stection [SectionNumber] from BDSVM/[FileName].svm and writes them to [dst].

Are you sure, trap_230 is pure XOR? The snapshots indicate that trap_230 is sometimes changing more than len (third parameter) times 4 byte (could also be some mistake in the snapshots / my calculations).
Yes it's pure Trap_Xor but i am currently not sure about the length.

The jump (BEQZ r6, 0x28) at 0x95064 (ic: 2691187) is not taken, but should be taken according to the PC-trace (r6 = 0xFF in my run).
Then you have a bug in your command dispatcher or and in at least one of the command handlers. Try to trace back why r06 is not zero. Btw I have heard reports of people who are already using snapshot number i (i>2000).

Edit: (removed)

Oopho2ei

15th September 2008, 18:36

The problem is the pc trace wasn't recorded at the same time as the snapshots. Through the traps "random" data is injected into the guest system which ultimately influences the instruction flow like you have experienced. So if that happens you either have to resynchronize your offset in the pc_trace.bin using the next snapshot data or simply don't use the pc trace at all. I haven't expected such non-deterministic behavior.

To provide evidence for this theory i made several pc traces and compared them. They all differ at offsets 01922478h, 019952D8h and higher. Please check if this matches your results.

I am sorry for all the confusion. It was my mistake (again) :rolleyes:

schluppo

16th September 2008, 01:23

I finished implementation of trap_230 (Trap_XorBlock) and trap_540 (Trap_MediaReadfile) (implementation is correct, according to Oopho's snapshots).

Next: Implementation of trap_110 (Trap_AES) cases 0xFFF10000 and 0xFFF10001.

frogman

16th September 2008, 03:48

QUOTE Oopho2ei. I haven't expected such non-deterministic behavior.
It's a Sony!
Looking good boy's. :p

schluppo

16th September 2008, 05:27

I finished the Implementation of trap_110 (Trap_AES) cases 0xFFF10000 (AES_Encrypting) and 0xFFF10001 (AES_Decrypting). :rolleyes:

This means, that the VM is now running correctly without using any snapshots for 2,081,361 instructions containing 477 trap calls. It initializes the device, obtains AES-keys, loads ~500kb from 00001.svm and then decrypts these ~500kb.

Dumping memory after the 477th call gives a snapshot that is identical to Oopho's post_trap_mem_476.bin (he starts counting at 0) and the registers are identical to post_trap_reg_476.bin at this point. Also the PC is in sync. :)

http://uploaded.to/?id=gn1mmh
Sorry for the terrible source.

derbeDeus

16th September 2008, 09:36

well, you didn't really implemented the aes trap for player keys, did u? ;) hehe
and i don't think discovery will work with other titles either; you see... there's a key in the data for dev1/qid3

Oopho2ei

16th September 2008, 16:49

I finished the Implementation of trap_110 (Trap_AES) cases 0xFFF10000 (AES_Encrypting) and 0xFFF10001 (AES_Decrypting). :rolleyes:
Looks good so far. Please don't treat the output of some traps as constants if you are not sure they really are. Also what currently matters is that (1) your trap implementation works for all occurrences in the snapshots and (2) that you extensively comment your code. It's not important that you write perfect java code. As long as the code is sufficiently commented more experienced programmers can quickly fix all the mistakes you made.

Also could you please add your modified version to our repository? The codebase from Disabled is currently broken and hasn't been updated for a while.

bmnot

16th September 2008, 19:23

Some suggestions:
- for branches you better show the absolute address and the not the relative address
- could you automatically load the snapshots?
- add a data view window (maybe double the number of columns for the register view and cut its height by half to make space for a small data view window below)
- reset the watchdog timer with ever trap call to 0x7FFFFFFF
- display the parameters of the trap call directly before it is executed (e.g. by commenting the values in the stack window)
- support of data breakpoints
- color the registers red which have been changed since the last step/run
- maybe assign some hotkeys
okay... some good ideas :) . These are implemented in version 0.2.0 of the Debugger: http://uploaded.to/?id=f0sxrm

Some notes:
- I found it somewhat ugly to hard-code the snapshot-names for auto-loading, so there's a menu option "Get post-trap snapshots from directory...". The Debugger will collect the files with the correct size in this directory and sort them alphabetically. It will load the next snapshot in the list whenever a trap was executed.
- There are read and write data breakpoints (displayed red and green). These can be set by selecting an area in the memory view and pressing the appropriate toolbar button.
- The traps including descriptions are parsed from a file that looks like this:

000010: void TRAP_Yield(UINT32 sleepTime); ?",
000020: UINT32 TRAP_EventGet(UINT8 *evBuf, UINT32 len); ?",
000110: UINT32 TRAP_Aes(UINT8 *dst, UINT8 *src, UINT32 len, UINT8 *key, UINT32 opOrKeyID);",
000120: UINT32 TRAP_PrivateKey(UINT8 *dst, UINT8 *src, UINT32 srcLen, UINT32 controlWord, UINT32 keyID); ?",
000130: UINT32 TRAP_Random(UINT8 *dst, UINT32 len);",
000140: UINT32 TRAP_Sha(UINT8 *dst, UINT8 *src, UINT32 len, UINT32 op); ?",
000210: UINT32 TRAP_AddWithCarry(UINT32 *dst, UINT32 *src, UINT32 len);",
000220: UINT8* TRAP_Memsearch(UINT8 *region, UINT32 regionLen, UINT8 *searchData, UINT32 searchDataLen); ?",
000230: void TRAP_XorBlock(UINT32 *dst, UINT32 *src, UINT32 len);",
000310: void TRAP_Memmove(UINT8 *dst, UINT8 *src, UINT32 len);",
000320: 5 parameters -> ?",
000330: void TRAP_Memset(UINT8 *dst, UINT8 fillvalue, UINT32 len);",
000410: UINT32 TRAP-SlotAttach(UINT32 slot, UINT32 codeLen, UINT32 reqPriority); ?",
000420: UINT32 TRAP-SlotRead(UINT8 *dst, UINT32 slot);",
000430: UINT32 TRAP_SlotWrite(UINT8 *newContents);",
000510: UINT32 TRAP_MediaReadRequest(UINT32 offsetHigh, UINT32 offsetLow, UINT32 len); ?",
000520: UINT32 TRAP_DeviceDiscovery(UINT32 dev, UINT32 qID, UINT8 *buf, UINT32 *len);",
000530: 3 parameters -> ?",
000540: 5 parameters -> ?",
000550: 6 parameters -> ?",
000560: 4 parameters -> ?",
000570: 0 parameters ... nop/vendor specific?",
008010: UINT32 TRAP_DebugLog(UINT8 *txt, UINT32 len);"

- The menu command "Load alternative trap descriptions..." allows loading modified such trap descriptions from a file. This can be useful if new information on traps is discovered - the Debugger will show updated, appropriate comments.
- Unfortunately, I haven't found time to look at the traps any further yet. This will be next. Hardening the VM with the 50000(!) :) snapshots is also interesting. Fortunately, schluppo already started on both :)
I integrated and slightly beautified his trap implementations (yesterday's version; hope I didn't break anything).
@schluppo It should be pretty straightforward to integrate your newest trap implementations.

The debugger is quite slow on inputting PC-traces. 4,000,000 PC values take 10min+.
Ok, this was clearly a flaw in the implementation - fixed that, too. Actually, this was only a minor detail:

File f = ...
/*before*/ for (int i = 0; i < f.length(); i++) { // hammering the file system
/*now */ for (int i = 0, l = (int)f.length(); i < l; i++) {
BTW, I could post more often. However, I am somewhat concerned about staying anonymous (although the activities should be perfectly legal here) - so I currently don't.

Oopho2ei

17th September 2008, 00:42

This is the output of aacskeys for this movie:MKBv: 4
Processing key: F190A1E8178D80643494394F8031D9C8
Encrypted C-value: 95247837F8A71FFDDBE8B0D3C2EE0575
Corresponding uv: 000000A0

Decrypted C-value: A174BE1DF3678FFCDBF8D43A7AF32B40
Media key: A174BE1DF3678FFCDBF8D43A7AF32BE0

Encrypted verification data: 810D5EE8B3011C4E8474E7702D12F55D
Decr verif data should be: 0123456789ABCDEF
Decrypted verification data: 0123456789ABCDEFD33493CD1DD12AC7

Drive FW info: 1.01

AGID: 00

Volume ID (AACSBYPASS): 1D2FC67462FDF986357A8F808FA1298B
Volume Unique Key: C5E868608ACD6700C2AC6A5E6AF533D9
Unit Key File Hash (DiscID): 6B63A40DD45F17A466B0C5CC35BF1500B4E0A9BA
Encrypted Unit Key 1: 6941CB89A6EEAB466AAB4B4F55DDA667

Decrypted Unit Key 1: 98E4D277F21AC464EFB0C0CA19FB50FD

I am posting this now because it's possible that some of these keys are used by the traps. It's unlikely that any of the important keys actually appear in memory though.
I have looked at the implementation of Trap_AES where the last parameter is a Key ID (in my particular case KeyID = 4). Like stated in the patent you have to first decrypt (aes-128-ecb) the encrypted aes key (pointed to by 4th parameter) using the player key and then use the resulting decrypted key to decrypt the data (second parameter) just like the last parameter were 0xFFF10001.
BUT the aes implementation doing the decryption of the key (first step) in my player has been transformed by some random mappings (different lookup tables basically) so the keys i could give you only work with the mapped aes version. Either i can figure out how to reverse the transformation or we have to use the mapped aes version for which i would provide a sample implementation in C. In the meantime you can play around with the keys above. Maybe it's one of them. :)

okay... some good ideas :) . These are implemented in version 0.2.0 of the Debugger: http://uploaded.to/?id=f0sxrm
Another great release. Thanks a lot! :)

tteich

17th September 2008, 10:07

PortableBDVM. (http://bdvm.100webspace.net/)

A friend asked me to post this since his account wasn't yet allowed to post.
Something's wrong. The site appears to be offline.

schluppo

17th September 2008, 15:34

Here's a new edit of the debugger:
http://uploaded.to/?id=vj3a9w

I took bmnot's newest version and changed the following:

Put return value of traps to R1 after trap_execution.
If a snapshot is loaded, compare result of trap_execution with the snapshot. Output nice error message in console, if there are any differences in memory or registers.
If you select a snaphot-directory containing at least 500 or so snapshots, "Run" will now run till trap 490, without loading snapshots (in order to run a bit faster - since the first 490 traps are treated correctly and since loading and comparing snapshots takes some time). From trap 490 on, snapshots will be loaded (and trap execution results will be compared to them).
Treatment of traps. The following is an exhaustive list of TRAPs that I encountered yet:

0x110: TRAP_Aes();
Cases AES_Decrypt and AES_Encrypt are implemented correctly and generically.
Case AES_Decrypt_Media did not occur yet but is an easy case, provided that we have the mediakey.
The other cases use player keys which i don't have and are thus not handled in a generic way yet.
0x140: TRAP_Sha();
The only case that occured yet (SHA_BLOCK) is handled correctly and generically.
Other cases are simple and can quickly be added if needed.
0x230: TRAP_XorBlock();
This is handled correctly and generically.
0x520: TRAP_DeviceDiscovery();
This is handled correctly, but with constant results.
For qId=0x3, the result seems to depend on a key that I don't have, so this trap is not treated generically yet.
0x530: TRAP_???();
This trap is concerned with getting some data from the player.
It is treated correctly, but not in a really generic way yet.
0x540: TRAP_MediaReadFile();
This is handled correctly and generically.

Summarizing on the traps: As long as they do not use keys that I don't have, they are treated in a (hopefully) generic way.

Note, that this version of the debugger is still getting out of sync with PC-trace at about 3000 instructions after trap #494 (TRAP_SHA). But even if no PC-trace is used, the post_snapshot #495 contains more differences than are justified by the execution of trap #495 (TRAP_deviceDiscovery(1,1,...)). Since the traps are treated correctly according to the snapshots, I come to the conclusion that there is a bug in the debugger's basic VM. I don't have the knowledge to find and fix it though.

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.

Edit: This version of the debugger is also in the repository now.

Oopho2ei

17th September 2008, 16:58

Here's a new edit of the debugger:
Thank you! :)

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.
I have asked someone who i think got past this point. This will hopefully get fixed soon.

This is a list of interesting details i was told anonymously:
- Trap_DeviceDiscovery, Trap_PrivateKey and Trap_AES use player specific keys
- the virtual machine is event driven but there is no "Trap_EventGet" (events are passed in a different way)
- Trap_010 is Trap_Finish
- the trap vector table in the 00000.svm is sorted alphabetically by trap names (all the traps are called from this table ... you know which one)
- Trap_520(1,3, 0x4514, 0x1EFF84) outputs the volume id at 0x4514+0x18 so question id '3' == 'what's is the volume id of the current media?'
- ...

A big thanks for sharing this information with us. :)

Oopho2ei

17th September 2008, 17:44

As long as this is not fixed, I can not get past trap #494 in a reasonable way, so it would be great if somebody could look into the execution after trap #494.
The person who got it running assumes your LB implementation is wrong and I was able to prove this :devil:.
Try the following: open the 00000.svm with a hex editor and overwrite the bytes at 0x1018 with the following sequence: "C8 00 10 04 88" This is cmd32 reading from 0x1004 (which is set to 0x88 by the last byte you replace). Now run the first command with the debugger. the result is stored in R0 and should be 0x88 but yours is 0xFFFFFF88. So somehow the byte is sign extended to integer.

Disabled

17th September 2008, 18:46

After a long time I'm reporting back. I stopped coding as the C99 and Java versions showed up, as I don't think there is a need for a third one.
Very good work of all of you contributing, including those anonymous ones!
On a side note, I fixed the "signed LB extension" bug in the svn. (Just remove the byte cast in L201.) I wasn't able to test it with the full run, as I don't have all the traces and files here, but it works on your test case and is now consistent with LW.
And I think it would be great to have more ppl hanging around the svn. At least bmnot should join it and I don't know who else is contributing or wants to contribute to the Java version or if there are more working on the C99 one they could of course use it too.

schluppo

17th September 2008, 19:01

Confirmed, removing the byte cast in L201 fixes this, now the debugger goes to trap #499 at instruction counter 6584939 with correct PC. :)

New traps Trap_Random (0x130), Trap_PrivateKey(0x120) and Trap-SlotRead(0x420) occur:

trap 494 is trap_520(1,1,0x1EFA94,0x1EF890->0x190); <works ok>

trap 495 is trap_140(0x1ef894,0x1efa94,7c,3); writes 0x7c bytes to 1ef894. <works ok>

trap 496 is trap_130(0x2348, 0x10); returns 0; puts 16 bytes to 0x2348: 2cb40ca4 65236776 0454806a 0eca1b92

trap 497 is trap_140(0x1ef924,0x1ef924,0x20,3); writes 0x20 bytes to 1ef924. <works ok>

trap 498 is trap_120(0, 0x5514, 0x2348, 0x10, 0); returns 0; writes 40 bytes to 5514:

4aa6ef81 dc4cf6b7 0237eaff e7d14a93
cfdc52e2 74811e16 2f770cfb ee11ce92
2a4e9fc4 beab67bd

trap 499 is trap_520(1, 1, 0x1ef794, 0x1ef794); <works ok>

PC goes out of sync between trap 499 and trap 500 due to nondeterministic behaviour.
Snaphot contents is ok though for at least the next 500 traps:

trap 500 is trap_110(0x5b430,0x5b430,?,?); returns 0; this case is not handled yet...

trap 501 is trap_520(1, 2, 0x1efcc8, 0x1efcc4); <works ok>

trap 502 is trap_520(1, 1, 0x1efe08, 0x1efcc4); <works ok>

trap 503 is trap_420(0x1efb14,0); returns 0. reads 256? bytes to 0xx1efb14.

trap 504 is trap_420(0x1efb14,1); returns 0. reads 256? bytes to 0xx1efb14.

trap 505 is trap_420(0x1efb14,2); returns 0; reads 256? bytes to 0xx1efb14.

the following 495 traps all read slots (there are 511 accessible slots, first 498 of them are read here).
Only the first slot contains data, the rest filled with 256? times 0x00.

Trap-Calls which are already handled correctly are indicated with "<works ok>" above...

Edit: New traps Trap_Slot-Attach (0x410), Trap_Slot-Write (0x430) appear...

Disabled

18th September 2008, 10:23

I took the adventure to download all the PC and Trap traces, about 1.5GB together. As I found that pretty inconvenient, I recompressed them using 7zip (http://www.7-zip.org/)and found they fit in less than 10MB! They still extract to ~20GB and are exactly the same as what Oopho2ei offered, but are much less on the download side. So here they are (http://www.zshare.net/download/18994624bff01fce/).
Under Windows I recommend extracting them to a compressed folder, then they will use just about 2GB of space and it will probably speed everything up, as the HD is the limiting factor for me. Linux users might use SquashFS or something similar.
As an extra and because I don't like uploaded.to that much I also uploaded the BDSVM folder here (http://www.zshare.net/download/1897968447300276/). *edit* This is the same as in post #130, just another server.

Oopho2ei

18th September 2008, 17:34

As I found that pretty inconvenient, I recompressed them using 7zip and found they fit in less than 10MB! They still extract to ~20GB and are exactly the same as what Oopho2ei offered, but are much less on the download side.
Awesome! I will make a package of the current movie ("The Day After Tomorrow") and of a "new" movie i have just bought ("I robot"). This time the instruction trace and the program counter trace will match the snapshots! :)

package structure:
/
AACS/
post_trap_snapshots/
BDSVM/
traces/
aacskeys_output.txt
title.txt
version.txt

Edit: i am currently compressing both packages so they should be available online in a few hours.

Edit:
Package 1 ("The Day After Tomorrow"): http://uploaded.to/?id=yodttu
Package 2 ("I robot"): http://uploaded.to/?id=mej65b