Finally handling BD+ (?)

[bshep]

So from these observations i can tell that TRAP_SlotAttach really allocates a new slot when called with parameter slot = 0xFFFFFFFF. But TRAP_SlotRead does not read the currently attached slot when called with parameter slot = 0xFFFFFFFF as described in the patent. Instead it returns three dwords which seem to describe what slot is currently attached and how many slots have been allocated or still can be allocated . Or something similar

This is just a wild guess but seems to fit:
DW0 = last allocated or currently attached slot ( you can test this by attaching a different slot and seeing if reading from slot 0xFFFFFFFF still give the last allocated slot or the currently attached slot )
DW1 and DW2 = seems to mean the number of slots available, would be interesting to see what happens if they are 0 and you try to allocate a new slot, if this asumption is true then 8 slots are avaiable to be allocated.

Edit: Just saw your edit, what does the test imply about DW1 and DW2, how many slots were already allocated in that test?

[Accident]

Sorry for the noise, but I have now got it running to this point:

[dlx] 0001A6F8 (I=18):ADDIE(R1  = R0  + 0007): 7 = 0 + 7
[dlx] 0001A6FC (I=38): SDW ([R29 + 10] = R1 ): 001EFF50 = 00000007
[dlx] 0001A700 (I=2D): JAL (R31 = 0001A704 ; PC += 04E8): 0001ABEC = PC + 1256
[dlx] 0001ABEC (I=39): TRAP([110]): TRAP 272
[dlx] 0001ABF0: TRAP 00000110 (272)
[dlx] TRAP_Aes()
[bdtest] executed 417 instructions

I am not sure it is all correct, but to verify I need to add TRAP_Aes first, since the snapshot is taken after the trap (correct yes?)

Since there is no stack, how are the arguments to TRAP_Aes passed? (or any trap for that matter).

(Edit1)
Ok, the param_check has details for at least Aes.
Here

(Edit2)
Although, not what to do with return code, but I will assume R1 as per DLX specification.

(Edit3)
My first trap to Aes return INVALID_PARAMETERS. So I need to do some debugging

Originally Posted by bshep

Edit: Just saw your edit, what does the test imply about DW1 and DW2, how many slots were already allocated in that test?

Well DW2 is decremented each time a new slot is allocated. It's initial value is 7 so no new slots were allocated at that time. I am not sure when happens when i try to allocate a new slot and DW2 = 2. My guess about DW1 is that it shows the number of the private slot that is currently allocated if you think of a few (~ 6) consecutive slots as private slots. DW2 seems to limit the number of additional slots the content code can allocate in in one session (when watching the movie once). I have to run further tests. If you have read something in the patents which matches above observations please tell me.

Originally Posted by Accident

Sorry for the noise, but I have now got it running to this point:

You should use the post-trap snapshots for all trap calls at the moment. You don't need to implement the parameter checking or the actual trap implementation yet. The stack pointer is (convention) r29 and the return value is stored in r01. Also you maybe don't want to send a message to the console for every instruction. When the initialization of the content code is complete (first call of TRAP_Finished) you will have executed millions of instructions and several thousands trap calls.
You should first make sure that the instruction processing is working by comparing your program counter, instruction register and the timer with the traces given in the packages you can find in posting #130. If you fixed all the bugs (if any) you should proceed implementing the parameter checking for all traps and verify with the post trap register snapshot that r01 was set correctly.

Edit: You have probably missed to look at these notes.

[Accident]

You should first make sure that the instruction processing is working by comparing your program counter, instruction register and the timer with the traces given in the packages you can find in posting #130

Can I confirm that 000000.dat is AFTER the first trap executes.

I can use the register.dat to confirm my instructions, sans R1 return code. So I will do that right now.

It only prints all instructions if I ask it to, so far I just tell it to run " 1 trap" and then it stops. I have a CVS, but what is the deal with "secrecy"? Is there need to stay anonymous? (Referring to Jon and CSS)

I had read those notes and forgot, I was following the wikipedia notes on DLX more My bad, apologies. Also, your instructions list for arithmatic shift has "see note below" without a note , but the wiki docs had it.

-----------------------------

Ok so the result is as expected, I guess it calls Aes with op=7 to test it. Please let me know if I am too noisy.

[dlx] 0001ABEC (I=39): TRAP([110]): TRAP 272
[dlx] ** TRAP 00000110 (272) return: 0001ABF0
[dlx] TRAP_Aes(): 80000001
[dlx] getPC (0001ABF0)
[bdtest] reached TRAP at 418: PC=0001ABF0.
[dlx] Comparing registers with 'dat/post_trap_snapshots/post_trap_reg_000000.bin'
Reg : VM Regs  == Loaded Regs
R0  : 00000000 == 00000000 ; OK
R1  : 80000001 == 80000001 ; OK
R2  : 001EFF6C == 001EFF6C ; OK
R3  : 00005550 == 00005550 ; OK
R4  : 00005570 == 00005570 ; OK
R5  : 00005560 == 00005560 ; OK
R6  : 00017A50 == 00017A50 ; OK
R7  : 00017A4C == 00017A4C ; OK
R8  : 00000000 == 00000000 ; OK
R9  : 00000000 == 00000000 ; OK
R10 : 00000000 == 00000000 ; OK
R11 : 00000000 == 00000000 ; OK
R12 : 00000000 == 00000000 ; OK
R13 : 00000000 == 00000000 ; OK
R14 : 00000000 == 00000000 ; OK
R15 : 00000000 == 00000000 ; OK
R16 : 00000000 == 00000000 ; OK
R17 : 00000000 == 00000000 ; OK
R18 : 00000000 == 00000000 ; OK
R19 : 00000000 == 00000000 ; OK
R20 : 00000000 == 00000000 ; OK
R21 : 00000000 == 00000000 ; OK
R22 : 00000000 == 00000000 ; OK
R23 : 00000000 == 00000000 ; OK
R24 : 00000000 == 00000000 ; OK
R25 : 00000000 == 00000000 ; OK
R26 : 00000000 == 00000000 ; OK
R27 : 00000000 == 00000000 ; OK
R28 : 00001160 == 00001160 ; OK
R29 : 001EFF40 == 001EFF40 ; OK
R30 : 001EFF84 == 001EFF84 ; OK
R31 : 0001A704 == 0001A704 ; OK
[bdtest] executed 418 instructions, 1 trap

Having fun so far..

Originally Posted by Accident

I have a CVS, but what is the deal with "secrecy"? Is there need to stay anonymous? (Referring to Jon and CSS)

Could you register at www.assembla.com and tell me your nickname so i can add you to the project. I make daily backups.
The goal of this project is to support BD+ in linux and other open source operating system so you can watch your movies without proprietary software on your pc. Reverse engineering to achieve interoperability is normally considered legal although patent owners usually try to fight this so you should stay anonymous.
Both AACS and BD+ were designed with renewability in mind so the system can quickly recover from a player compromise and leaked key material which is a major advantage over css. The key material which i provide for debugging will very likely soon be revoked so this implementation will very likely stop working for newly released movies. So this project is no threat to the security of the system. But both AACS and BD+ can only renew protection for movies released after the compromise.
Finally Slysoft has successfully reverse engineer BD+ a while ago and they sell their product commercially.

Originally Posted by Accident

Please let me know if I am too noisy.

It's the purpose of this thread to guide the implementation efforts. It's likely that more people have the same questions. If you don't spam the thread with new postings it's fine. You can also write me private messages.

Edit: revision 86 now has experimental support for proper slot handling. @Schluppo and others: please run some tests

Still:

#002037 TRAP-SlotRead          (001EFD08, FFFFFFFF);
Register error at trap number 2037:
post_reg_snapshot:
00000000 00000000 00000000 10020000 00000101 00000001 0002282C 001EF9DC
001EFADC 00000100 00000000 0003E01C 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00001160 001D02CC 001EFD1C 0007C4D4
post_reg_execute:
00000000 80000001 00000000 10020000 00000101 00000001 0002282C 001EF9DC
001EFADC 00000100 00000000 0003E01C 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00001160 001D02CC 001EFD1C 0007C4D4
Byte 1EFD09: snapshot: 00 execute: 02
Byte 1EFD0A: snapshot: 00 execute: 28
Byte 1EFD0B: snapshot: 00 execute: 2C
Byte 1EFD13: snapshot: 07 execute: 00
Memory Error at trap 2037

All other calls of TRAP-Slot* behave as in the snapshots ("DAT v1.02") with the revision 86 of the debugger.

#002037 TRAP-SlotRead          (001EFD08, FFFFFFFF);

All the time is was looking for a reason why this call should fail and why our debugger lets it execute successfully.
The constant 0xFFFFFFFF gets signed extended when it is compared with a variable of type long. I made the same mistake several times so the last patch (to revision 87) fixes multiple bugs. Could you please check how the debugger behaves now after TRAP_Finished when you pass events (eg. write the event id and parameter manually to 0x00-0x030)?

OK, until the first TRAP_Finished, only TRAP_Random, TRAP_DeviceDiscovery(1,3) and TRAP_PrivateKey give different results than the snapshots (due to 'random' data in these results).

I ran the debugger without guidance to the first break and then invoked some events with following results:

event 0x110(0,0xFF000) - TRAP_0x20 and then TRAP_Finished are executed. This event behaves as in the snapshot-package.
event 0x210(0,1) - Just TRAP_Finished is executed. This event behaves NOT as expected.
event 0x110(0,1) - Executes the whole bunch of trap testing. This event behaves as in the snapshot-package.
event 0x220(0,1,n) - Just TRAP_Finished is executed. This even behaves NOT as expected.

So, why do some events behave differently than in the snapshots?

1) Either there is an unnoticed fault/bug in the trap-treatment leading to a 'bad' state of memory when the first TRAP_Finished is reached. This bug could only be in the three traps which are introducing random data (since the other traps behave as in the snapshots).
2) Or the behaviour of the events could depend on the order in which they are executed. This would mean, that for instance event 0x220(0,1,n) can not be invoked unless event 0x110(0,1) has happened earlier. It seems to be somehow reasonable that BD+ first wants the traps to be tested before it is willing to decrypt movie data. Nevertheless, I tried and first let the debugger run to the first TRAP_Finished and then fed event 0x110(0,0xFF000) to the code. Then I invoked event 0x210(0,1) on the resulting state. This is the same order of events as in the snapshots and still event 0x210(0,1) behaved bad.
3) I may have made a mistake while doing these tests. I will double-check my method of event-invoking now.

Thanks. I will make a longer snapshot package now. In the meantime please look at figure 5 and 6 in this document and read the corresponding text (if you haven't done that already).

Edit: A new package (v1.03) for "The Day After Tomorrow" is available now: http://uploaded.to/?id=5dwn30

I compressed the snapshots with the diffarchive tool i introduced a few weeks ago. It was written by one of our anonymous contributers and manages to shrink the 50GB of snapshots to less than 20MB. There is no need to extract the archives on disc although the tool provides you with that option.

For every archive i've included the first snapshot so you can test if you handle the archive properly. The first snapshot is always compared against the empty snapshot (filled with zeros). A description about the archive structure can be found here. I also wrote a small readme which contains the syntax in EBNF.

@Schluppo: i am currently implementing support for the archives in the debugger as a new class "snapshots". Four instances (objects) of this class "snapshot_trap_mem", "snapshot_trap_reg", "snapshot_break_mem" and "snapshot_break_reg" for every snapshot type we have. The constructor gets the filename on instantiation and eg. with "snapshot_break_reg.next()" would you switch to the next snapshot. The current snapshot memory can be accessed with eg. "snapshot_break_reg.memory". I hope it get it working soon you can start your analysis of the event handling

[Accident]

It was going well, until trap milestone 484 showed I forgot instruction 0x1E.

Are there any saves of what the PC should be at each post_trap_mem milestone? My program runs on its on, and the registers all compare correctly (except R1 from the trap), it fixes R1, loads the next core and resumes.

But somewhere along the line the PC becomes very repetitive, until eventually it gets stuck on trap PC=0001AC74 (looping back to the trap). It stays here until post_trap_mem_002029.bin where PC goes really high (somewhere in the 3GB area, handled with the address masking, but there are only zero's there). So I must be doing something wrong earlier perhaps, which could be that my PC is lost.

I also do nothing with WD yet, apart from initialising it. I assume I should still be able to finish all the post_trap_mem milestones?

Boring output:

[bdtest] reached TRAP at 37019913:2026 PC=0001AC74.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002026.bin'
[bdtest] reached TRAP at 37019929:2027 PC=0001AC74.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002027.bin'
[bdtest] reached TRAP at 37021630:2028 PC=0001AC04.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002028.bin'
[trap] trap_DeviceDiscovery()
[bdtest] reached TRAP at 37023357:2029 PC=0001ABFC.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002029.bin'
[dlx] FE80103D (I=2C): J   (PC += FFFFFFFC): FE80103D = FE801041 + FFFFFFFC
[endless loop]

Edit:
I wonder if it is related to port 164 but checking my JMP, I get the following values:

    U_Jimm = I & 0x3FFFFFF;             // 25 bit constant                      
    S_Jimm = (I & 0x2000000) ? ( (I & 0x3ffffff) | 0xFC000000) : (I & 0x3FFFFFF);

U_Jimm 0245AE89 S_Jimm FE45AE89
[dlx] 001EFC6C (I=2C): J   (PC += FE45AE89): 0024AAF8 = 001EFC70 + -28987767

If Unsigned IMM is 0245AE89, then Signed Imm extended to 32bit becomes FE45AE89. Then if PC is 001EFC70, the out come is indeed: 0024AAF8.

Originally Posted by Accident

Are there any saves of what the PC should be at each post_trap_mem milestone? My program runs on its on, and the registers all compare correctly (except R1 from the trap), it fixes R1, loads the next core and resumes.

You are supposed to use the traces of the program counter the instruction and the watchdog timer to verify that you are executing the right instructions in the correct order.

Originally Posted by Accident

If Unsigned IMM is 0245AE89, then Signed Imm extended to 32bit becomes FE45AE89. Then if PC is 001EFC70, the out come is indeed: 0024AAF8.

Yes, sign extend the immediate value and then add the result to the pc and apply the mask (0x3FFFFC).

[Accident]

Originally Posted by Oopho2ei

You are supposed to use the traces of the program counter the instruction and the watchdog timer to verify that you are executing the right instructions in the correct order.

Well, admittedly the Registers would only compare correctly to the saved files if the PC was, in fact, already correct. I was trying to find possible reasons as why it fails. So I guess it is running correctly. I just don't know why it would get stuck at trap 2029. But perhaps it sits there waiting for WD to expire or something, since I do nothing with WD. Should I decrease it every instruction or something similar?

Nor do I use INSTF.

I am glad to announce that the debugger seems to handle the new differential post-trap snapshot archives correctly. It is also considerably faster now:

Originally Posted by Accident

But perhaps it sits there waiting for WD to expire or something, since I do nothing with WD. Should I decrease it every instruction or something similar?

Yes, by the number of cycles the instruction needs. You can find the table in posting #25. You won't need it until after the first call of TRAP_Finished().

Originally Posted by Accident

Nor do I use INSTF.

The instruction filter is set by cmd3A and you have to xor it with the instruction to get the decrypted instruction.

If you need help finding your bug write me a private message with the details (where did the pc desynchronize, differences between your memory and the snapshots, ...)

[Accident]

Originally Posted by Oopho2ei

Yes, by the number of cycles the instruction needs. You can find the table in posting #25. You won't need it until after the first call of TRAP_Finished().

You know, I really should print the TRAP number:

[dlx] ** TRAP 00000010 (16) return: 0000103C
[bdtest] reached TRAP at 37236320:2676 PC=00001038.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002676.bin'
[bdtest] executed 37236320 instructions, 2677 traps (-1)

Which just happens to be TRAP_Finished(). heh oh dear...

The instruction filter is set by cmd3A and you have to xor it with the instruction to get the decrypted instruction.

Perfect, that is all I need to know to implement that, and WD.


Edit:

[dlx] ** TRAP 00000010 (16) return: 0000103C
[bdtest] reached TRAP at 37236320:2676 PC=00001038.
[bdtest] loading 'dat/post_trap_snapshots/post_trap_mem_002676.bin'
[bdtest] break reached, PC=00001038: WD=FFFFFFFF
[bdtest] executed 1110978144 instructions, 2677 traps (2)

Ok, INSTF, WD and breaks are done. Should I go back and work on Traps, or carry on with Breaks?

Edit 2:

It isn't that I am ignoring your advice Oopho2ei, but Aes I understand, and is trivial to implement, where as I have no idea what a DeviceDiscover, or Slot, is so I am leaving those to later. Aes has been done.

[dlx] ** TRAP 00000110 (272) return PC=0001ABF0
[trap] trap_Aes(AES_DECRYPT_PLAYERKEYS): 0x2156dc->0x2156dc (3 key 4)
[bdtest] reached TRAP at 4290:5 PC=0001ABEC.
[dlx] comparing with core 'dat/post_trap_snapshots/post_trap_mem_000005.bin'
+VM.addr
-loadedcore
[dlx] compare complete. 0 differences.

Edit2:
Amusingly, MediaReadFile() should have taken 20 mins, but took all day. Looked like it was trying to read beyond the end of the memory, so I made a much more complicated read_file function. Turned out I hadn't got the right level of len ptr to ptr to ptr. Sigh.

Originally Posted by Accident

Ok, INSTF, WD and breaks are done. Should I go back and work on Traps, or carry on with Breaks?

You should start working on the traps now. The group number of a trap is "trapID >> 8". I suggest you implement the groups in the following order: 2 -> 3 -> 4 -> 5 -> 1 -> 0 so you start with something easy like TRAP_XorBlock and advance to more difficult traps like the cryptography traps in group 1. You shouldn't start with events/instruction breaks because we don't fully understand this yet. If you think you can handle the snapshots pretty well you should consider to update your code to the new diffarchive snapshots (see posting #249 for details). The archives are much smaller and faster to work with but more difficult to implement.

I've made a package containing the latest debugger binaries, the snapshots of "The Day After Tomorrow" in the new much smaller diffarchive packages, the traces and all the other small files needed to run the content code.

all the files needed to run the machine and to study events: Full debugger package (extracted: 672MB)

Simply extract this archive in a directory and run the debugger from the root of this directory (java -jar Debugger.jar). It should find all the files it needs automatically so there is no need to select anything from the menu. You only have to click run and watch the console output. The debugger stops on every execution break so the changes applied during the breaks to the virtual machine by the player can be examined. When you have studied the output press run again to continue until the next break. When a difference between snapshots/traces and the real state of our machine is detected a warning is sent to the console and the state of our machine is automatically adjusted. So don't worry.

You should consider to redirect the console output to a file (eg run with "java -jar Debugger.jar > output.txt") because it's quite a lot of data.

Edit: of course it's "mismatch" and not "missmatch". Sorry for that copy/paste typo.

Edit: TRAP_Schluppo is trap#0020. I gave this trap a (temporary) name so people (Schluppo and others...) stop using the former (wrong) name "TRAP_EventGet". I was told that "TRAP_EventGet" as described in the patent doesn't exist. TRAP_Schluppo might be the trap which passes the results of the content code to the player. The player then uses this data to repair sections of the audio/video stream.

Edit: if you rename the files/directories so the debugger can't find the traces/snapshots the debugger will run without guidance. This is only useful for developers to check where things start to get bad.

The obfuscation scheme guarding the player AES keys has been broken by one of our anonymous contributers. You can find the plain keys in posting #163.

This will surely ease implementation efforts. Great work!!!

My obfuscated AES implementation is now obsolete

Originally Posted by Accident

It isn't that I am ignoring your advice Oopho2ei, but Aes I understand, and is trivial to implement, where as I have no idea what a DeviceDiscover, or Slot, is so I am leaving those to later. Aes has been done.

TRAP_Aes was difficult to implement until a few hours ago. Now you no longer need "my" obfuscated aes algorithm so it's much easier.

There has to be some kind of firewall around the virtual machine which validates all communication between the ( potentially hostile ) content code and the outside world (traps and events). Part of the rules which are enforced by that firewall are the parameter checks on every trap call. It's obvious that the traps and the event handling itself has to be carefully implemented. I believe this additional effort is necessary to prevent the content code from breaking out of it's sandboxed environment and do nasty things like gathering user information and "calling home" when it detects an unlicensed emulator. So because these additional security measures make things more difficult i suggested to test this code first with the easy traps. This is a draft of our debugger design (the interface contains the firewall).

Originally Posted by Accident

Amusingly, MediaReadFile() should have taken 20 mins, but took all day. Looked like it was trying to read beyond the end of the memory, so I made a much more complicated read_file function. Turned out I hadn't got the right level of len ptr to ptr to ptr. Sigh.

Are you willing to provide us with your latest source code so we can have a look how it's going? Sadly you ignored my offer to use our assembla repository.

I have uploaded the two certificates we get from TRAP_DeviceDiscovery. You can view them here. The currently unknown data section is supposed to contain:

- certificate version
- player manufacturer identifier
- serial number
- signature date and time
- device (player) attribute information

In the above order (source p.40 / figure 11). Maybe someone can identify pieces of the unknown data reading the corresponding text of the patent.

Edit: These are the original certificates of the player. Nothing has been modified.

[Accident]

Originally Posted by Oopho2ei

TRAP_Aes was difficult to implement until a few hours ago. Now you no longer need "my" obfuscated aes algorithm so it's much easier.

This is true, I simply call OpenSSL now, thanks to that. All of what I have done is based entirely on Your findings, I have yet had the chance to do any engineering of my own.

Are you willing to provide us with your latest source code so we can have a look how it's going? Sadly you ignored my offer to use our assembla repository.

Not ignored, but at the time, my code really didn't do any more than your sample so it felt pointless. But I am reaching a stage where it might lead somewhere, so I will register on assembla now.

Edit: Registration completed, same nick as here.