Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
21st October 2008, 01:54 | #261 | Link | |
Guest
Posts: n/a
|
Well i provide the raw data and other people analyze it together with me. If necessary they ask for more details (mostly via private messages). It's working pretty well and finally i am the one who presents the results. I surely did my part but it's only fair to say that other people who try to stay anonymous have contributed to this project a lot and i am thankful for it!
Quote:
We only have 200MB disk space so try to avoid uploading binaries. You can commit your changes as often as you like. |
|
21st October 2008, 07:14 | #262 | Link |
Guest
Posts: n/a
|
OK, just so that the obvious was stated in the thread: The new snapshots contain an additional (second) call of event 0x110(0,1) - followed by the usual bunch of trap-checking. After this event, the VM finally seems to be in a state to be willing to do the decrypting of the movie. Lots of calls of event 0x220(0,1,n) follow. And for each invocation of event 0x220, the VM just seems to test different traps (as noted in postings #211 and #213 - note that I made the experiments from these two posts without snapshot-guidance and still the results were quite correct). After the trap-testing, each event 0x220 is treated with a TRAP-SlotAttach(0,14) (I don't know whether successful or not) and a TRAP-SlotRead(0,*) afterwards. However, the data which is read from slot 0 is always the same, so this is certainly no 'input-channel'. Two calls of TRAP_DeviceDiscovery get the two certificates before the final TRAP_Finished ends the treatment of the current event.
Furthermore, all the interruptions just change the event-id and event-parameters and nothing besides, so that the first 0x1000 bytes of the VM's memory are no 'input-channel' either. Also I observed that (when restricting attention to the first 0x1000 bytes) the VM is not changing more than the first 0x3F bytes during treatment of event 0x220, meaning that also the output needs to happen in some other way (64 bytes is surely not enough output for 30 seconds or so of stream-repairing). Currently I am guessing, that all the traps that are being executed after event 0x220 was invoked are just concerned with verifying that the VM is still running unmodified. I further guess that the VM is not using traps to communicate the important data with the player. One of the patents mentions a conversion table, so maybe the parameter <n> of event 0x220 is just used to copy a specific part of this table to a predefined area inside the VM's memory and then the player can later access this table on it's own and replace the bad data in the video stream with the appriopriate entry in the currently 'active' part of the conversion table. This would mean, that there is apart from parameter <n>, no real data-input to the important event. An another note, there are still bugs left in the treatment of at least TRAP_DeviceDiscovery, TRAP_AES and TRAP_MemSearch. More information about that soon. Also TRAP_DiscoveryRAM is still problematic (seems like parts of the RAM change over time after all :/) Edit: The details about the bugs: Code:
#004433 TRAP_DeviceDiscovery (00000001, 00000001, 003FFD48, 001EF810); [!] register missmatch after trap call: r01: 00000000 (vm) != 80000001 (snapshot) Code:
#004006 TRAP_DeviceDiscovery (00000002, 00000000, 001EE854, 001EE7D0); [!] memory missmatch after trap call: 1EE854: 52 (vm) != 00 (snapshot) [!] memory missmatch after trap call: 1EE857: 00 (vm) != 52 (snapshot) [!] memory missmatch after trap call: 1EE858: 38 (vm) != 07 (snapshot) [!] memory missmatch after trap call: 1EE859: 04 (vm) != 80 (snapshot) [!] memory missmatch after trap call: 1EE85A: 80 (vm) != 04 (snapshot) [!] memory missmatch after trap call: 1EE85B: 07 (vm) != 38 (snapshot) [!] memory missmatch after trap call: 1EE860: 04 (vm) != 00 (snapshot) [!] memory missmatch after trap call: 1EE863: 00 (vm) != 04 (snapshot) [!] memory missmatch after trap call: 1EE864: 01 (vm) != 00 (snapshot) [!] memory missmatch after trap call: 1EE867: 00 (vm) != 01 (snapshot) Code:
#008136 TRAP_MemSearch (001CFA58, 00000014, 001CFA6A, 00000002, 001CF998); [!] memory missmatch after trap call: 1CF999: 00 (vm) != 1C (snapshot) [!] memory missmatch after trap call: 1CF99A: 00 (vm) != FA (snapshot) [!] memory missmatch after trap call: 1CF99B: 00 (vm) != 6A (snapshot) Code:
#007861 TRAP_Aes (001EE210, 001EE220, 00000000, 001EE1C0, FFF10000); [!] register missmatch after trap call: r01: 00000000 (vm) != 80FFFFFF (snapshot) Edit3: Just a wild idea: Could this trap here: Code:
#002677 TRAP_Schluppo (00000000, 00001000); Edit4: I found that after (and before) execution of Code:
#007426 TRAP_Schluppo (000FCD9E, 000EFAF8); Code:
<header> then lots of blocks, where a block looks like this: 32 bit = <blocklen> <blocklen> * 32-bit address (?) - increasing all the time ~1,5 * <blocklen> bytes - correct data for the according address? Last edited by schluppo; 21st October 2008 at 10:53. |
21st October 2008, 14:09 | #263 | Link |
Registered User
Join Date: Aug 2002
Posts: 111
|
If I may ask, what should the VM do when WD expires (we get a break?) I've been perusing the thread, but I am still not certain. Some java mention "events" that perform:
Code:
vm->R[28] = vm->PC; vm->PC = 0x1000; vm->WD = 0x7FFFFFFF; |
21st October 2008, 14:58 | #264 | Link | ||
Guest
Posts: n/a
|
Quote:
This topic is still under research so you shouldn't start to implement this yet. What we currently think happens is that when the content code calls TRAP_Finished it signals to the player that the virtual machine has finished all tasks and is now idle. The watchdog timer is now set to 4000 which causes the "idle loop" of the content code to be interrupted every 4000 cycles. On every interruption the player checks an event buffer ("mailbox") for new events. If there is an event then the vm memory section 0x00-0x40 is modified and the registers are changed as you described. If there is no event then only the watchdog is reset to 4000 and execution continues in the idle loop. A watchdog timer value of 0x7FFFFFFF basically means "don't interrupt the execution" whereas a value of 4000 means "do event polling". Every call of TRAP_Finished restores the program counter from register 28 and sets the watchdog timer to 4000. Quote:
There has been an "Accident" with our repository which i could repair with the backup of yesterday evening so you need to checkout the entire repository again if you updated today because revision 97-99 (currently) don't exist. Last edited by Oopho2ei; 21st October 2008 at 15:10. |
||
21st October 2008, 19:29 | #265 | Link |
Guest
Posts: n/a
|
Some details about TRAP_DiscoveryRAM and the free space pattern:
Code:
cd f0 a2 e2 ed 88 e3 98 bc 66 c1 7c f4 db 47 96 f6 6c 5e 11 af 2f 40 42 41 07 de 4c 8a 63 4d 51 c0 9b 38 27 19 03 97 65 3d 44 ac a7 18 a0 61 13 b3 b4 c6 21 15 e0 c5 0f 78 c4 ef 2c 53 26 2e 67 54 5f d5 c8 aa 17 46 95 a3 94 fb ba d3 bd 64 2a bf 34 48 35 43 d7 f5 cf 90 92 2d b5 5d 93 99 50 74 72 31 04 58 10 5a 7f ff ca 55 37 b2 dd e5 0a 0d 69 b1 3a 00 3c ea 22 32 8d f2 9c 86 1c b0 76 30 01 d2 06 bb 77 f3 80 e8 a6 05 ec 89 49 fd d9 d6 d4 45 6f 4f b8 33 57 d8 87 a8 9e f9 5c 23 b6 6b eb 7e 1f 02 fe 85 e9 12 c9 ae 08 9f 52 25 71 09 3f 29 68 3b 1a e7 91 59 7a 6e 8e 56 a4 1d 1e a9 9a df 70 8c 4e 4b dc ee 36 8f c3 83 82 e4 8b 79 6d 0c a1 7d 39 4a be fa ab d1 c7 28 7b cc f7 b7 ad 62 b9 d0 f8 f1 0e 1b cb da e6 2b 60 16 c2 81 0b 73 a5 20 84 5b 24 9d 75 e1 ce 14 6a 3e fc Code:
private byte[] BuildVirtualPlayerMemorySection( int src, int len ) { byte[] buffer = new byte[len]; int address; // fill the section with free space pattern for (int i = 0; i < len; i++) { address = ( src + i ) & 0xFF; buffer[i] = constants.FREESPACE_PATTERN_LUT[ ( address * address ) & 0xFF ]; } // write real player data if there is any for ( int i = 0; i < len; i++ ) { address = src + i; if ( address >= 0x00000000 && address <= 0x000001FF ) // 0x00000000-0x000001FF : player name buffer[i] = playerName[ address ]; if ( address >= 0x00000200 && address <= 0x0000021F ) // 0x00000200-0x0000021F: player version buffer[i] = playerVersion[ address - 0x200 ]; if ( address >= 0x00000220 && address <= 0x0000031F ) // 0x00000220-0x0000031F: unknown player info buffer[i] = playerInfo[ address - 0x220 ]; // TODO: there is more } return buffer; } Some parts of the virtual player memory seems to contain real data at fixed addresses. The (undefined) rest of it is filled with this pattern. I still don't know if this pattern is just garbage data or required by the specification. You should be able to find the answer when you set data breakpoints on this pattern and run the debugger. Schluppo: Could you also look at TRAP_DiscoveryRAM in revision 97 (and later)? As you can see the RAM.bin is no longer used instead everything is filled with this pattern and later partly overwritten with contents we know (see above). You will see from the console output which calls of TRAP_DiscoveryRAM don't return the pattern. The first difference is at this call: Code:
#002151 TRAP_DiscoveryRAM (0026D976, 001EFC94, 00000080); Edit: This is the latest binary of the debugger. Simply replace the "Debugger.jar" of the package from posting #256 with this one. It's faster but the changes in the implementation of TRAP_DiscoveryRAM causes a lot of warnings to be sent to the console now. You may delete the RAM.bin and the crypto directory because these files are no longer needed. Last edited by Oopho2ei; 21st October 2008 at 20:59. |
22nd October 2008, 00:50 | #266 | Link | |
Registered User
Join Date: Aug 2002
Posts: 111
|
Quote:
trap_finished(): WD=0x4000, PC=R28 (even the first time?) VM runs endless loop, until WD expires R28=PC, WD=4000, PC=? instructions executes trap_finished() called again.. WD=0x4000, PC=R28... ? ----- Either way, I will return to filling in the TRAP code for now, I have quite a few to go. I will also glance at the diffarchives, since waiting 2600 traps takes a while. ---- Edit: I can't get any post_break_mem snapshots to work, probably as I don't know what PC to set, I have implemented all the Traps that make sense to me. But I am still missing PrivateKey, *Slot (wtf is a slot?) traps. Not entirely sure what to do next. Last edited by Accident; 22nd October 2008 at 12:36. |
|
22nd October 2008, 09:24 | #267 | Link | |||
Guest
Posts: n/a
|
Quote:
If the "mailbox" is empty then nothing happens. Execution continues without any changes. The interesting case is when really an event occurred so the "mailbox" is not empty. Then the PC is so set to 0x1000 (always?) and r28 and WD are set as you described. Quote:
Quote:
The DAT v1.03 package contains 10929 post-trap snapshots and 1816 post-break snapshots which altogether need less than 11MB disc space. |
|||
22nd October 2008, 18:26 | #268 | Link |
Guest
Posts: n/a
|
We have identified the remaining address ranges of the virtual player memory which contain real data. You can view the updates here. It looks like most of the data is changing over time.
Edit: the different pattern in section 00250000 - 0028FFFF is created like this: Code:
if ( address >= 0x00250000 && address <= 0x0028FFFF ) // 00250000 - 0028FFFF : pattern similar to freespace pattern buffer[i] ^= (int) ( ( 3 * address * address + 1 ) & 0xFF ); Last edited by Oopho2ei; 23rd October 2008 at 12:45. |
23rd October 2008, 05:13 | #269 | Link |
Guest
Posts: n/a
|
TRAP_DiscoveryRAM is giving lots of problems now (seems the implementation is not finished yet). Any idea how to treat the areas of memory that are changing over time?
Event handling is actually quite easy - I think, there is the following standard program that all the players try to invoke on the BD+ -VM: Code:
intialize + auto-test VM event 0x0110 (0,0xFFFF) (reset conversion-table) event 0x0210 (0,1) (check whether player is ready for playback) event 0x0110 (0,1) (test VM and load conversion table) event 0x0110 (0,1) (test VM and load conversion table) event 0x0220 (0,1,1) (partially test VM) event 0x0220 (0,1,2) (partially test VM) event 0x0220 (0,1,3) (partially test VM) ... event 0x0220 (0,1,n) (partially test VM) event 0x0010 (shut down) However, since our VM is doing all the work (once it finally works 100% correct), we can just invoke the events in this order and get the data needed for decoding the movie from the conversion table. I think, that the non-event interruptions can be omited altogether (at least for the first 4 "intializing" events), since we don't have to wait for user-(or player-)input when we just want to decrypt the movie. Anyway, I am getting the impression, that the player is not just taking the data from the conversion table and then putting it to the right place in the stream. I rather think, that there is some algorithm being executed on the data from the conversion table before the "correct" data for the stream is obtained. I found out, that for every entry in the conversion table, there are 16 bytes assigned to it (AES-key, anyone?) If you have access to the player's routines, it would be helpful to know, what the player is doing with the data that it is (hopefully?) reading from the conversion table (the player should read from the conversion table whenever it encounters 5 "bad" bytes in the stream. The conversion table for "DATv1.03" is laying in VM-memory at 0xEFAF8 after finished treatment of event #4). Also it is still necessary to find out the connection between the addressing-value in the conversion table and the actual position of the "bad" location on the disk / inside a file (the first entry in the conversion table is holding address-value 0x0009090A and assigns the following 16 bytes to it: 0xB6BE742B 0x32C794D0 0xBCCCB272 0x76B8AF89 - to what is 0x0009090A referring and how are the 16 bytes used to get correct data for this block?). Finally, here is some more info on the conversion table found in "DATv1.03" (correcting some faults from my last posting): Code:
Start-Offset: 0x000EFAF8. 0x000EFAF8: 00 00 24 00 53 01 01 00 // unknown - table-length? Then 0x0153 32-bit adresses. // block-adresses? 0x000F004C: 00 00 00 00 // unknown 0x000F0050: 82 00 00 00 // block 1 length Then 0x0082 32-bit adresses. // block 1 row 1 Then 0x0082 times 16 bytes. // block 1 row 2 0x000F0A7C: D0 00 00 00 // block 2 length Then 0x00D0 32-bit adresses. // block 2 row 1 Then 0x00D0 times 16 bytes. // block 2 row 2 etc... Then when all blocks are finished: 0x001EC424: 02 00 00 00 0F 00 34 00 Then 0x0034 32-bit values in the following pattern (n = 0..C): 0F 00 n2 CA 0F 00 n6 CA 0F 00 nA CA 0F 00 nE CA The table seems to end at 0x0001EC4FB Edit2: Just an observation: When using snapshots and outputting all the trap-logs etc., the debugger's VM is running around 5 times slower on my computer, than the VM on the player that was used to create the snapshots. I hope this is not creating any timing problems (BD+ code could be expecting certain things to be finished in a specific time-window or something similar). Last edited by schluppo; 23rd October 2008 at 05:47. |
23rd October 2008, 10:05 | #270 | Link | |
Registered User
Join Date: Feb 2007
Posts: 85
|
Quote:
|
|
23rd October 2008, 12:38 | #271 | Link | |||
Guest
Posts: n/a
|
Quote:
Quote:
Yes i have seen how the table (which is stored at the beginning of 00002.svm) is parsed and i also saw those 16 bytes blocks being decrypted by xor with some other values which possibly come from the VM memory too. It would make sense if we need other data from the content code to work with this conversion table because otherwise we wouldn't need to execute the content code. Quote:
Edit: The content code seem to modify these 16 byte blocks after loading it from 00002.svm Code:
#005051 TRAP_MediaReadFile (00098DEC, 00000000, 00000000, 000EFAF4, 000EFAF8); // read 0x000EFAF4 bytes from section 0 of 00002.svm and store them at 0x000EFAF8 ... // decrypt table contents? ... #005052 TRAP_Schluppo (000FCD9E, 000EFAF8); // send result (0x000FCD9E bytes at 0x000EFAF8) to player Edit: That's the final table which is kept in player memory during playback: conversion table? Edit: Another note on performance. The C implementation of Accident will probably be much faster so i don't think this will be an issue. Last edited by Oopho2ei; 23rd October 2008 at 18:13. |
|||
25th October 2008, 17:15 | #273 | Link |
Guest
Posts: n/a
|
To give you an idea of what is coming: content repair code.
That's the code which does the actual replacement of the media stream based on some data structures created from the conversion table entry: "1d 50 c1 91 f0 2f 4c c8 43 21 5d f9 f3 68 f1 05". The first array 'A' is a constant (for one bad spot in the media stream) whereas the second array 'B' is different each time and so are the shift lengths which itself are calculated from 'v0'. So this basically means 'B' is actually an array of 6 bytes with each entry expanded from 8 bytes to 16 bit by placing the 8 bit at a random position in a random 16 bit value. This looks like obfuscation to me. Will all the obfuscation removed this code probably collapses to a simple xor loop of two 5 byte arrays Btw. If you look at byte position 6 and 7 of the conversion table entry (see above) you will find 0x4C and 0xC8 which also occur at the end of the array 'A' which is no accident. Edit: Have a look at posting #13 for more details. Last edited by Oopho2ei; 25th October 2008 at 17:51. |
25th October 2008, 17:57 | #274 | Link | |
Registered User
Join Date: Apr 2007
Posts: 7
|
Quote:
Also B[0] doesnt seem to be used for anything. So it seems that the data in A[] is the corrupted data somehow modified and B[] is the 'key' information obfuscated with some random data for which v0 is the 'seed'. I agree that once the obfuscation algorithm is understood this can probably be collapsed to a simple XOR. Last edited by bshep; 25th October 2008 at 18:04. |
|
25th October 2008, 18:55 | #275 | Link |
Guest
Posts: n/a
|
Just to confirm everyones guess about the origin of A:
Code:
<---- ????? ----> <---- A ----> <---- ?? ----> E = 1d 50 c1 91 f0 2f 4c c8 43 21 5d f9 f3 68 f1 05 A = 21 43 5D C8 4C A[0x00] = E[0x09] = 0x21 A[0x01] = E[0x08] = 0x43 A[0x02] = E[0x0A] = 0x5D A[0x03] = E[0x07] = 0xC8 A[0x04] = E[0x06] = 0x4C Note that E was already transformed by the player when every entry of the table (sent by the content code via TRAP_Schluppo) was xored with "AB EE B5 BA C2 E8 D8 18 FF ED FF 8B 85 D0 5E 8C". Also the first 6 bytes of E (as above) are transformed to "4A 1A A0 53 64 B1" this is why i believe E consists of at least 3 parts. @bshep: don't worry about v0 and B[0]. Your guess about A and B may be right. Last edited by Oopho2ei; 25th October 2008 at 19:08. |
25th October 2008, 21:09 | #276 | Link |
Guest
Posts: n/a
|
One of our anonymous contributers figured out that the certificates are signed with a 1280bit RSA signature and also found the public key of the license administration.
Code:
the public key is (n,e) with e = 3 and n = 8B169F529C28B5D45DB5D1607B831BED31381D38AEF561A43E744326DD00765E E7A47F353D4A8C507752B08A6671259AAF140E86EEB1D05D344EF801A5AFB150 3A82BE089DCF25618852199D26CC79AE99466A231999AAC6C26E7DDA662304A7 72D1B304C9CD0C724434D640E29BE64FBBE1E7993A30939D6FB925AE0C350896 14F89FBAE9B931FC01D4D10732EB62CA8878E1894BD82F3007806D75CE172B57 Code:
signature from certificate 1: 03B04675B5AAB8B1E66974F1C90667C846C7CDF5259F9E95139BF12D6A46312F 7575E28404ABA98914EEE83A65F752EFCFA115D1145BC605AAFD68D9287D4F2F 99536F014ED21717871B2B7CF3299499BBBA8AF5D459AEE240A314D51AE4F991 D445585C14E291041C4358B337CF9F14E48699DF291A5CB571CB61FD68ACD7E6 1EC2FD1296884E0F95683760929B8EB34009C69974E839E2E4B63F75FAF3726F SHA-1 Hash of certificate 1: 77A325225B91CEF17DCD6F1785016A9FFFFD4C8E result of encrypting the signature with the public key: 0001FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00302130 0906052B0E03021A0500041477A325225B91CEF17DCD6F1785016A9FFFFD4C8E Great work again. PS: if you don't know how to encrypt a message with RSA it is simply: result = message ^ e mod n |
25th October 2008, 21:47 | #277 | Link |
Guest
Posts: n/a
|
I was given a OCR pdf version of patent US 2007/0033419 Al. This might be helpful if you want to search the document and see the pictures.
|
26th October 2008, 02:21 | #278 | Link |
Guest
Posts: n/a
|
I have looked at the code which creates 'B' and i saw that "AB EE B5 BA C2 E8 D8 18 FF ED FF 8B 85 D0 5E 8C" which is initially used to transform every entry of the conversion table (fix-up table) is used here too.
This is something i did not expect and it might be that this xoring with "AB EE B5 ..." is not part of the preparations to use the conversion table. Instead it might be an attempt to make it more difficult to steal this precious table from the players memory using various memory dumping techniques. It surely is an easy target because the conversion table has to be present in memory during playback. So my suggestion is to continue working with the table as it passed from the content code to the player with TRAP_Schluppo. One more thing: It looks like 'B' only changes around every 20-30s (telling from the memory write accesses over a certain period of time). I could also confirm this experimentally. If i change B i get a lot of distortions (video freeze over several second) whereas if i change A only a few frames look bad. Edit: There might also be a relationship between the number of entries in a block of the conversion table (eg. first block has 0x82 entries) and the number of "bad spots" in one video segment. Last edited by Oopho2ei; 26th October 2008 at 02:29. |
26th October 2008, 07:39 | #279 | Link |
Registered User
Join Date: Apr 2006
Posts: 2
|
If you would like, I have a SSL-Protected SVN server (with access control) that could be used. I own the box personally, and have a good relationship with the ISP - in the event of a DMCA takedown notice, or similar, I would have an opportunity to take advantage of the counter-takedown provisions of the DMCA. Many ISPs do not provide notice, and simply shut down the box.
|
26th October 2008, 12:21 | #280 | Link |
Guest
Posts: n/a
|
Thanks. I am thinking about a new program which repairs all the "bad spots" in the media stream using the conversion table and which i won't upload to our repository. The debugger can be used to generate this table but apart from this it is only useful to study the content code so i hope it is legal and doesn't infringe any patents. I am also hoping that this project doesn't draw any attention in the media because we don't do any better than commercial tools like AnyDVD-HD except that we support platforms other than Microsoft WIndows.
|
Thread Tools | Search this Thread |
Display Modes | |
|
|