Jump to content
Tuts 4 You

Help with reverse engineering abandon project's assets for use as


Jeremy__

Recommended Posts

Sorry if this is the wrong section, but there doesn't seem to be a section for this sort of project.


 


I am at very mature stages in a game development project and I hope to gather a few team members at one of the up-coming full indie conferences to really turn the recent demos into an actual game. The game I have in mind has a graphical theme so akin to the Dark Eden project. I've tried a couple place holder art but I really have trouble finding a sufficient collection of assets to complete a good 'concept design' of the game. So I've decided to write a concept game as a mod of Dark Eden, and then move away from my dependency on their assets as I find an intrigued artist.


 


Anyways, so I've been trying to reverse engineer the file format used by Dark Eden. I've pretty much concluded most of the loading of assets, handling of rendering etc abstracted away by a library IFC22.dll (just by the number of exports and their signatures.)


 


As a programmer I am fairly competent (but again, I have moved to managed environments [Java/C#] since...) as a reverse engineer I'm more than rusty. I know some of the basics...


 


Anyways, there are two approaches I can use to recover the graphical assets. The first is likely the more difficult method, and that is to entirely reverse engineer their file format (or sufficiently) and then parse them and generated the graphical assets. The second I believe would be significantly easier. The second is to gain sufficient knowledge of the asset handling interfaces in IFC22.dll to load the assets to memory and then render them on to a graphics surface and dump them and whatever meta-data may accompany them...


 


Alright, but doing any static analysis on how IFC22.dll is used is difficult since the protection used on the executable is hiding some of the imports. (I totally forget how to reconstruct an IAT, but I am sure I could figure it out.) From static analysis I would probably move on to just hacking around with the exports in some C++ test-bed.


 


I believe it is using VMProtect or EXECryptor. The executable exports the symbols:



Name Address Ordinal
---- ------- -------
EXECryptor_AntiDebug() 005234CA 1
EXECryptor_DecryptStr(x,x) 0052336E 2
EXECryptor_EncryptStr(x,x) 00523350 3
EXECryptor_GetDate() 00523309 4
EXECryptor_GetHardwareID() 0052333F 5
EXECryptor_GetProcAddr(x,x) 005234B0 6
EXECryptor_GetTrialDaysLeft(x) 0052338C 7
EXECryptor_GetTrialRunsLeft(x) 005233A1 8
EXECryptor_MessageBoxA(x,x,x,x) 00523490 9
EXECryptor_ProtectImport() 005234D9 10
EXECryptor_SecureRead(x,x) 00523413 11
EXECryptor_SecureWrite(x,x) 005233B6 12
start 0061F248

I haven't noticed any junk code (in terms of realigning instructions etc) but there are some messy obstructions here and there (but they're pretty easy to ignore.) There doesn't seem to be any active ani-debug implementations either.


 


Can someone provide any tips in terms of recovering the IAT or otherwise reverseing their sprite format? Thanks.


Edited by Jeremy__
Link to comment

In general, analyzing file format is easier and less messy than rendering assets and dumping them from graphics surface. 

But in your case, I'd start with ready-made tools. 

 

Like here: http://forum.ragezone.com/f857/darkeden-tools-701784/ and here: http://www.progamercity.net/game-files/3260-tool-darkeden-spk-packer-extractor.html

And then - wherever Google takes me. :)

EDIT: fixed formatting and broken links. oops.

Edited by kao
Link to comment

Hey Kao,


 


I really appreciate those links. I must've been googling the wrong kind of stuff (Ie, file format documentation opposed tools!) Makes me feel terrible at googling.


 


Anyway, I need to automate the exporting/translating of assets so it would still be nice to get some documentation on the file format - that said reverse engineering an unprotected SPK editor to determine the format is significantly simpler than using the Dark Eden executable! I'll give it a shot thanks.


 


Thanks again.


Link to comment

Alright, so after several hours studying one of the unprotected SPK Extractor tools I've finally discovered (at least to a pretty good degree of confidence, I have yet to actually apply the knowledge into writing an extractor...) the specifics on the SPK and SPKI file formats. I imagine that they don't differ too much (ie, minus some meta-data) when it comes to the sprite files etc. I will document my findings on those file formats as well.

 

The SPK and SPKI file formats are actually incredibly simple. In hind-sight, it may have just been easier to study the file format alone... Anyways...

 

Essentially, anything that isn't animated is stored in an SPK file. All the world tiles, all the world objects etc. Each SPK file has an associated SPKI file. Ie, the file "tile.spk" has the associated spki file of "tile.spki" . The SPKI  document is just an array of offsets from the origin of the associated SPK file that points to the compressed data of an image.

 

DarkEden SPKI Format

 

First two bytes form a 2-byte short (little-endian) that is equal to the total number of graphics in the stored in the associated Graphic Archive File (associated SPK file.)

 

Following this is an array of dwords, where each entry in the array represents an offset from the origin of the associated Graphic Archive File (SPK) which contains the associated compressed graphic.

 

 
DarkEden SPK Format
 
Each compressed graphic inside of the SPK (pointed to by the SPKI) will decompress into a 16bit BI_BITFIELDS compressed bitmap minus the header (Ie, just bitmap data/pixel data.) The header is very easy to reconstruct (in-fact, it never changes with the exception of a couple fields.)
 

The graphic, in compressed form, will be in the following form. 

 

struct
{
WORD imgWidth;
WORD imgHeight;
[Array of compressed 'chunks']
}

 

Each chunk is structured like so

struct
{
WORD chunkSizeWords; //Excluding header
WORD isLineFilled;
WORD data[chunkSizeWords]; //omitted if isLineFilled is false
}

 

 

Each chunk represents a single compressed horizontal array of pixels. Ie, an image that is 5x10 will have ten chunks. If isChunkFilled is zero (false) then the entire chunk is blank (white, that is it extracts to an 0xFFFF run equal to the width of the image.) If it is true, then following the chunk is a a series of 'streams.'

 

The first stream encountered in a chunk is always a fill stream. It consists of a single WORD that defines the number of 0xFFFF runs to extract. The second stream is a copy stream and it consists of a single WORD and a series of bytes. The word in this case is the number of words that must be extracted from the following array of words. The following stream is a fill stream, then a copy stream etc etc, continuing to alternate.

 

You may encounter a fill stream with a zero-length. ie 0x0000. All this does is toggle from fill to copy with no effect. For example if you were you compress the following stream:

 

0x1234 0xFFFF 0xFFFFF 0xFFFF 0x1234

 

it would compress into

 

0x0000 0x0001 0x1234 0x0003 0x0001 0x1234

 

Since the first stream is interpreted as a fill stream.

 

Another example,

 

0x0002 0x0002 0xAABB 0xCCDD 0x0003

 

Would extract to

0xFFFF 0xFFFF 0xAABB 0xCCDD 0xFFFF 0xFFFF 0xFFFF

 

After you've extracted the pixel data, all you need to do to make it a valid bitmap file is to prefix it with the appropriate header. Here is a header I've constructed

42 4D 36 1F 00 00 00 00 00 00 46 00 00 00 28 00 00 00 C6 00 00 00 EC FF FF FF 01 00 10 00 03 00 00 00 F0 1E 00 00 12 0B 00 00 12 0B 00 00 00 00 00 00 00 00 00 00 00 F8 00 00 E0 07 00 00 1F 00 00 00 00 00 00 00

Note though that you'll need to adjust the bfSize (DWORD at offset 0x2), the image width (DWORD at offset 0x12), and the image height (DWORD at offset 0x16). Additionally, you will need to adjust the biSizeImage field (DWORD at offset 0x22) since that is the size of the image's pixel data.

 

Note that the bitmap format specifies that if the pixel data describes the image in a top-down fashion then the height must be negative. This is the case for all spk graphics. So the height field is a signed long equal to negative the image height in pixels.

 

Anyway, I will continue to look into the sspk file format. Hopefully it doesn't differ too much.

Link to comment

Alright, I had today off and looked into the file format a bit more. There is definitely a lot of meta-data stored in the ispk files that I have not fully understood how to extract. I assume the cspk documents also contain meta-data I have yet to understand. However, I do know how to extract the bitmap images from ispk documents. It is the same algorithm described above _except_ rather than alternating from fill, copy, fill, copy etc... the pattern is fill, copy, copy, fill, copy, copy. Which is very interesting to me, because optimally those two copy operations would simply be merged into one. Instead, there is a separation. I've observed the separation across 'materials.' For example, take a look at the character head in the below image:


 


http://imgur.com/JOq2DQN


 


The copy pixel operation for the skin is different than the copy pixel operation for the hair, the armor etc etc. I think that there is some sort of pixel to material mapping encoded over the image. For example, the first copy would be for pixels that correspond to character hair. The second copy is for pixels that correspond to player skin. The third copy would be for player armor. Idk... still working on the idea from that angle...


 


That variable that I thought was "isLineFilled" has actually become more interesting. The thing is, in none of the file formats I've encountered has there been an isLineFilled member equal to zero (false.) They've all been equal to 1 or greater. In the UI components this value is always one. In the SPK extractor I was examining, all that was done was to check if this value was zero (if it was, the entire line was encoded as blank.) However, that flow of execution was never taken by the spk extractor on any of the spks I extracted using it. Leading me to believe that this variable actually hints at something more interesting, and the behavior responding to it in the spk extractors I examined was a guess made by the programmer.


Edited by Jeremy__
Link to comment
  • 2 weeks later...

After banging my head against my keyboard for a few hours... I have reverse engineered the cfpk format. The respective cfpki format just indexes into individual artifact animation sets in the cfpk file (ie, you can ignore it, unless you need to quickly parse the cfpk file...) Here is a basic run-down of the file structure (cfpk.) From it you can extract the necessary meta-data to reconstruct and translate character animations - which was the ultimate goal of this project.


 


I am going to integrate a utility in the open-source project to extract animations. 



typedef struct frame_t
{
short spki;
short offsetX;
short offsetY;
} FRAME; typedef struct frameSet_t
{
short numFrames;
FRAME frames[numFrames] <optimize=false>;
} FRAMESET; typedef struct animation_t
{
BYTE numPerspectives;
FRAMESET perspectives[numPerspectives] <optimize=false>;
} ANIMATION; typedef struct artifactAnimationSet_t
{
byte numAnimations;
ANIMATION animations[numAnimations] <optimize=false>;
} ARTIFACT_ANIMATION_SET; typedef struct file_t
{
short numArtifacts;
ARTIFACT_ANIMATION_SET artifactAnimations[numArtifacts] <optimize=false>;
} FILE; FILE file;

Edited by Jeremy__
Link to comment

Current revision in the github repository now supports extracting artifact animations and exports frame/animation meta-data into a nice & easily parse-able json document. This is pretty much the end of the project. I might go a little further and try to extract the material maps out of the ispk documents at a later time -- but that would be a fairly small addition.


 


The project is open sourced at: https://github.com/JeremyWildsmith/darkedenkit


Edited by Jeremy__
  • Like 1
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...