Playing with API Implementations with IDA and Bochs

It is common, when one reverser keeps talking to another, to find out new ways to use tools we already have. Sometimes, you find out how to use the tool in an unexpected way, others is just an unknown shortcut or undocumented behaviour. Lately, this had often happened while talking about using IDA and Bochs together, specially using the debugger in the special "PE mode". Bochs is one of the common environments where it is being used to run/analyze malware, we found interesting to show what you can do, what you can't and what really needs to be improved in this new debugging mode.

Just to put some context, in the current version of IDA Pro (right now is 5.5 as I wrote this), Hex-Rays included a new debugger plugin so it is possible to debug targets using Bochs x86 emulator. This new plugin allows three different ways to debug the targets:

- Disk image: You can use an image or "bochs virtual machine" to debug your target. It's probably the best way but it requires having a working virtual machine so for those who never tried, it's a slightly painful process.

- IDB: its intended use is only to select a piece of assembly code and debug it "virtually" using Bochs. Think about it like an advanced x86emu.

- PE: Quite similar to the IDB mode but where IDA provides a basic environment with a PE loader, and support for the emulation of some win32 API calls. How this emulation works and how we can play with it is the main topic of this article.

This article is focused on the 'PE mode", so let's take a look about how it works.