SendMessage in a debugger in another thread

[Killboy]

Hi,

I was trying to inject a dll into a process and use WM_COPYDATA for sending the data needed for the dll to work.

The problem is that I'm in the middle of the debug loop, ie the remote thread on LoadLibrary doesn't return until I loop through WaitForDebugEvent/ContinueDebugEvent.

So I called it in another thread, and found something weird happening:

When I call SendMessage (logging something to a listbox) in the thread, it doesn't return and seems to hang up.

At the same time, WaitForDebugEvent hangs up as well, and doesn't return (it's set to INFINITE).

I found this at MSDN but I'm probably too retarded to get it...

Messages sent between threads are processed only when the receiving thread executes message retrieval code. The sending thread is blocked until the receiving thread processes the message. However, the sending thread will process incoming nonqueued messages while waiting for its message to be processed. To prevent this, use SendMessageTimeout with SMTO_BLOCK set.

The thread sending the messages waits for SendMessage to return but why does WaitForDebug end up in an infinite loop ?

I know I can catch the thread exit message in the debugger, but thats too much hassle. I just want all the stuff in one function and not use like 10 additional global vars.

Edited January 3, 2009 by Killboy

[ap0x]

I had this problem and solution came from reversing ImpRec. It turns out that one antidebugging trick is very useful in this situation. So here is what you do. Create a suspended thread, use ZwSetInformationThread(hThread, 0x11, NULL, NULL) to hide it from the debugger and it will run as soon as you call ResumeThread. Then WaitForSingleObject will work as it should. Neat trick, a?

[Killboy]

I was going to solve it without any hacks ^^ Thanks though, will try that if I don't get it done the other way :/

I think I found the actual problem, I missed the important remark at MSDN

If you call SendMessage from inside the debug loop (if the loop is in the same thread you called DialogBox from) it just calls the message handler. If you call it from a different thread, it sends the message and activates the thread but doesn't return until SendMessage returned, even if theyre both in the same process.

I created another thread that contains the whole debug stuff, ie

main thread = dialog box + handler

second thread = debugger

third thread = dll injection

this seems to work, it's a bit weird though, I can't wait for any of the threads with WaitForSingleObject obviously as it would lock up the initial thread :/